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SUMMARY 


The overall goal was the development of a Cockpit Ocular Recording System (CORS). 
This effort was broken into four tasks: 

• Development of the system, 

• Experimentation and improvement of the system, 

• Demonstration of the working system, and 

• Documentation. 

The System 

Overall the prototype represents a workable and flexibly designed CORS system. For the 
most part, the hardware used for the prototype system is off-the-shelf. The single exception is 
the flight data acquisition unit emulator (FDAU-M) which was designed, developed, and specially 
built. These hardware items include: 

• A helmet mounted oculometer with a video recording feature, 

• A Polhemus magnetic head position sensor, 

• A personal computer with a parallel processing and a serial board added, 

• A flight data acquisition unit emulator, 

• A digital flight data recorder, and 

A time code generator is also used to provide timing signals for the video 
recording and purposes of validation. 

All of the following software was developed specifically for this effort: 

• The setup software permits the user to enter the cockpit configuration and to 
identify dwell regions for use by the CORS system, both in processing and 
playback. 

• Three conceptually different functions are performed within the CORS 
computer during data collection. These are: 

- The sensing software which integrates the 60Hz data from the 
helmet mounted oculometer and the head sensing units, 

- The processing software which uses the newly formed position 
signals, applies a spatiotemporal filter and determines fixation/dwell 
position, 

The results are sent out to the recording medium which may be the 
screen, a disk file, or the flight data recorder. 

• The playback software allows the user to retrieve the recorded data and 
examine them in the context of the regions identified with the setup routine. 
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Experimentation and Design Improvement 


Several experiments are reported which were carried out to test the system accuracy. 
These experiments show CORS is a viable concept and also highlight some areas where 
component improvements can be made. The areas of improvement identified include: 

. The calibration routines which cannot be modified within the current 
oculometer. On occasion, the calibration routines preclude obtaining good 

results. 

. The helmet and method of mounting the cameras and visor on the helmet. 

There is considerable movement of the helmet on the head and of these 
devices on the helmet. 

These improvements can be made and several suggestions are provided in the conclusion 
section. 


Demonstration of the Concept 

A demonstration experiment was reported which showed modest accuracy of the system. 
Additionally, software was developed for the CORS computer to work in conjunction with the flight 
data acquisition unit emulator and the flight data recorder to write and retrieve data. This effort was 

successful. 




INTRODUCTION 


Eye scan data provide an important clue in understanding an operator's acquisition of 
information, visual workload, strategies, and the associated human pertormance. With the advent 
of non-intrusive eye movement recording devices, specilicall, the oculometer. the leasttnlrtyol 
obtaining such data has .ncreased dramatically. The development ol the technology has 
prompted a number of practical applications in which eye scan data can be used in a variety of 
ways: evaluating instrument design, as a substitute lor manual manipulated, analysis ot behavior 
for training, accident investigations, and a host of other applications. 

The potential benefits ot developing and using an oculometer for training, and especially 
accident investigation, resulted in the present prelect. This report describes the results ot 
Analytics' ellorts to develop a prototype Cockpit Ocular Recording System (CORS) for recording 
pilot eye scan data on a FAA approved Digital Flight Data Recorder. The work was pedormed 
under a Phase II Small Business Innovative Research contract sponsored by NASA Langley 
Research Center, contract NASI -18473. 

The wont described in the present report is predicated on the results ot the Phase l ettod 
and it will be useful theretore to review, briefly, the issues and results ot that study. 


Background - Phase I 

The objective of the Phase I effort was to investigate the concept of collecting and 
recording eye movement data on a digital flight data recorder (DFDR). In Phase I, a number of 
technical issues reviewed and analyzed. 

. Is it technically feasible to collect eye movement data in a cockpit environment 
using an oculometer? 

• Is the modern DFDR capable of recording processed oculometer data? 

• How useful is pilot eye movement data in accident investigation and 
reconstructions? 

• What other uses exist for pilot eye movement data gathered during actual 
flight operations? 

These issues were investigated in detail and the results were presented in a repod (Arnold 
Deimler, & LaGrossa. 1986, Analytics Technical Repod 2042), In summary the findings were: 

. Components ot commercially available oculometer systems exist that (with 
modification) seem appropriate for the intended application. 

• Digital flight data recorders have the capacity to store processed output 
signals. 
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• The inform a tion recorded on the DFDR should have extensive utility. 

Based on these findings, the concept was judged to be feasible and warranted development, 
experimentation, and demonstration. The results of this effort are documented in this report. 


Purpose and Goals of the Phase II Project 


The overall goal ol the Phase II project was to develop a prototype to demonstrate the 
feasibility of recording eye position data on a DFDR. The specific approach to meeting the overall 
objective is contained in the following steps: 


• Build a fully functional prototype laboratory system. 

Conduct tests to determine system response characteristics. 

• Refine and optimize system configuration. 


nnl S ' 9 l a r, d C ° ndU< ? T PihCal studies t0 ^derstand the technical and 
perational issues of collecting and recording eye movement information on 
a commercial flight data recorder in a "cockpit-like" environment. 


• Demonstrate the utility of the system. 

• Prepare system documentation of the prototype design. 

The first five steps have been accomplished and are documented in this report. 


Organization of the Report 


The report is divided into a number of sections. These sections are organized such that 
each of the three tasks as defined in the statement of work is discussed in a separate section. 
General material is provided in other separate sections. The sections are: 


• Section 2 provides an overview of CORS, including 
architecture and a brief description of the software 
components. 


the functional 
and hardware 


• Section 3 contains a discussion of Task 1 which included 
integrating the overall system. 


developing 


and 
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Section 4 presents the results of Task 2 which involved conducting 
experimental studies to test the initial system and providing suggestions for 
improvement of the algorithms. 

Section 5 is a discussion of some recommendations and suggestions for the 
future. 

Appendix A is a glossary providing definitions of the terms used. 

Appendix B contains the data from the initial experiment. 

Appendix C contains an up to date review of available eye scan technology. 
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OVERVIEW: SYSTEM CONFIGURATION 


The purpose of this section is to provide an overview of the CORS system. Here we 
provide the basic architecture of CORS along with some comments about the function of the 

various pieces. Details on the algorithms, hardware, software, and functions will be found in the 
next section. 

The primary function of CORS is the collection and recording of eye scan data, stated in 
terms of positions and instalments scanned in the cockpit. The use of CORS involves more than 
just data collection, however. One way to consider CORS is in terms of the sequence of activities 
in the overall application. There are three steps as shown in Figure 2-1 : 

Step 1 : Setup in which the user defines the various parameters, planes, 

surfaces and instrument locations, in a cockpit or in a workstation. 

• Step 2: Runtime in which the actual data collection, processing, and 

recording of eye scan data is performed. 

Step 3: Playback with which the retrieval and analysis of the eye scan data is 

performed at some time after the actual data collection. 


Step 1 
Step 2 

Step 3 


SETUP 


SENSING 


PROCESSING 


RECORDING 


PLAYBACK 


Figure 2-1 . Simple diagram showing the relation of the steps and the operations 
performed within each step of CORS. 

The setup step is required before any data collection can be performed. This involves giving the 
CORS system the necessary information about the location of instruments in the cockpit. Once 
this is accomplished, it is possible to perform the runtime step. 

There are three main functions in Step 2, runtime, of CORS: 

• Sensing, 

• Processing, and 

• Recording. 
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These three functions are shown across the top of Figure 2-2. The main portion of the figure 
shows the principal hardware components. The sensing is performed by the oculometer (Eye 
View Monitor) and the head tracker (Polhemus) sensing devices represented on the left side of 
the figure. These signals are passed into the CORS computer (represented in the middle of the 
figure) which performs the processing necessary to determine points of gaze and to convert 
these eye position signals into real world coordinates (fixations and dwells). The resulting fully 
processed signals, representing scan data, are then passed to the flight data acquisition unit 
(FDAU) emulator, shown on the right side of the figure, which transforms the data into the format 
necessary for the digital flight data recorder (DFDR). 

Once the data have been written on the DFDR, the FDAU emulator is also used to 
perform the third step, that of retrieving the data and sending them back to the computer for 
playback, display, and analysis. 
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CORS PC 
(COMPAQ) 



Figure 2-2. Relations between the sensing, processing, and recording portions of CORS runtime. 









TASK 1: DEVELOP AND INTEGRATE SYSTEM 


The overall goal In Task 1 Is: To develop the prototype system. 

Equipment will be acquired and hardware will be Integrated to 
perform according to the design goals and required capabilities. 
Software will be developed for subcomponent Interfaces. 
Subcomponent processing and computational software will be 
developed for data processing, data compression, and system 
control. The result will be a functional CORS system that will be 
further refined through experimental studies conducted In Task 2. 

Devetoping the prototype consisted of three separate activities. The first consisted of 
acquiring off-the-shelf hardware and assembling these components. Much of the Task 1 project 
effort was centered on the second activity which consisted of tying these components together 
functionally by developing software. These software components included the setup, runtime, 
and playback routines. The third activity consisted of developing interface hardware to convert 
signals between the CORS computer and the DFDR formats. 

In the following sections, we discuss the various hardware and software components in 
terms of Figure 2-2, moving from left to right. After discussing the runtime prototype, we will then 
consider the system software, including the setup and playback steps illustrated in Figure 2-1. 


The Sensing Components 


OASIS Eye View Monitor (EVM) Description 

The original OASIS (Zaklad, et al„ 1986, Analytics Technical Report No. 1977) eye/voice 
testbed relied on an Applied Science Laboratories (ASL) Eye View Monitor System (EVM), Model 
1996. The EVM is a remote head oculometer. That is, an infrared (IR) source and a pupil camera 
are both located in a housing which is positioned, on a fixed base, several feet from the subject. 
On the basis of the IR image captured by the pupil camera, the EVM calculates pupil and comeal 
reflection centroids. In turn based on these two centroids and their relationships as established 
during calibration, the EVM calculates a point in a plane which exists at a fixed distance from the 
subject's eye. Although the eye turns and points to different locations in the plane, the eye (and 
therefore also the head) is presumed to be in a relatively fixed location with reference to the plane. 
The EVM sends x and y eye position and pupil diameter, to the host computer (Masscomp) at a 
rate of 60Hz. 
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CORS Upgrade 


In the original OASIS EVM, the subject's eye was required to be within a cubic inch 
volume throughout the calibration and test run. The remote system can be upgraded with mirror 
tracking to allow for a cubic foot of eye/head movement. However, the head movement 
restrictions required by a remote oculometer system cannot be tolerated in an operational setting. 
Accordingly, this alternative was discarded not only for the continued limitation on eye/head 
movement but also for the space limitations created by a cockpit for installing the associated 
devices. Instead, the EVM Model 1996 was retrofitted with a helmet mounted oculometer and a 
magnetic Polhemus head tracker was added to the system. 

Helmet Mounted Oculometer (HMO) 

In Phase II, the remote head oculometer was upgraded to an HMO. This was done to 
permit the full range of head motion which would normally be found in an operational setting such 
as a cockpit. The equipment is mounted on a large regulation motorcycle helmet. This insures 
that the full range of experimental subjects can use the helmet, but unfortunately this does not 

preclude the helmet from moving on the subject's head for people with smaller head sizes than 
the helmet. 

Four components are mounted on the helmet: the pupil camera, the visor, the scene 
camera, and the illuminator. The pupil camera is mounted on the helmet directly above the 
subject's eye. The pupil camera can be moved laterally to center the pupil horizontally in the field 
of view (FOV) of the camera. The visor is used to center the pupil vertically in the FOV of the pupil 
camera. The pupil camera has a 20° FOV with reference to eye rotation and as long as the pupil 
stays within this FOV the system can function correctly. However, the normal eye movement 
range of the subject is up to approximately 30° (± 15°) of eye rotation and glances outside the 
FOV of the camera are lost and cannot be recorded. The pupil camera also has a small depth of 

focus which requires that the optical path length from the camera to the pupil be constrained to a 
relatively constant distance. 

The visor is a transparent piece of plexiglass which has two films laminated onto the visor. 
The first film reflects infrared illumination and is used to monitor the pupil and cornea. The second 
film is a one way mirror allowing the subject to look through the film, while allowing the scene 
camera to view what the subject is looking at. Two telescoping arms attach the visor to the helmet 
and there is a joint at the helmet to allow the visor to be shifted away from the helmet. This allows 
the subject to remove the helmet easily without the visor being in the way. Additionally, the joints 
and the telescoping arms allow the visor to be positioned at the correct angle and the correct 
optical path length. Generally, this means that the visor is about four inches from the subject's 
pupil and at approximately a 45° angle. With repeated use the visor tends to slip during rotation of 
the head upwards or downwards and will gradually move out position during use. 
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A provision is available for video recording of the scene through the scene camera. This 
tape can be viewed at a later time to compare the results provided by the computer against the 
scene viewed by the test subject at the time of testing. The scene camera is mounted on an arm 
which is attached to the side of the helmet and then is positioned underneath the visor and 
directly in front of the subject's nose and mouth. The scene being viewed by the subject is 
reflected off the visor and onto the scene camera. Before the subject is calibrated, the scene 
camera must be positioned such that the internal representation of the oculometer calibration 
points agrees with the image being transmitted by the scene camera. This is crucial since after the 
subject's head is unrestrained the only external corroboration of the CORS results comes from 
the data collected from the scene camera. If the camera is not in the correct position the results 
shown on the scene camera monitor are inaccurate and misleading. Obtaining the correct location 
can be a difficult task since ihe arm that the scene camera is attached to has 7 joints. This is a 
complicated assembly with many degrees of freedom and. like the visor, continued use has 
loosened the joints and the assembly tends to move if there is any head motion. This made it very 
difficult to conduct a verification of the head motion algorithms. 

The final component of the helmet assembly is the illuminator. The illuminator is mounted 
next to the pupil camera and contains an infrared LED. The infrared light is reflected from the visor 
into the subject's pupil. The level of illumination is controlled by the eye view monitor and is 
adjustable. The illuminator is generally nonintrusive and the subject notices only a slight drying of 

the eye. 

Head Tracker System Description 

The Polhemus Navigation Sciences' 3-Space Tracker, which was also added to the CORS 
system, provides the data necessary to map EVM coordinates into world coordinates as the 
subject moves about. The system consists of a magnetic source suspended above and behind 
the subject and a magnetic sensor attached to the top back of the subject's helmet. The tracker 
can provide Cartesian coordinates and orientation data in one of three forms: 1) orientation 
angles, 2) direction cosines, or 3) quaternions. For the sake of computational efficiency, the 
CORS algorithms are configured to use quaternions. 

The 3-Space Tracking system utilizes low-frequency, magnetic field technology to 
determine the position and orientation of a sensor in relation to a source reference frame, 
providing six degrees of freedom for the movement measurement device. The source unit is 
mounted on a frame located behind the helmet and the sensor is mounted on the helmet; head 
motion is calculated from variations in the magnetic fields as measured by the sensor. 

The primary operational range of the headtracker is a spacing of 4 to 28 inches between 
the source and the sensor. The headtracker can resolve a positional change of 0.03 inches and 
an angular change of 0.1°. The system has a positional static accuracy of 0.1 RMS inches and an 
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angular static accuracy of 0.5°. Like the EVM eye tracker, the headtracker outputs data at a rate of 
60 times/second with a single source/sensor pair. 


Algorithmic Integration of Helmet Mounted Oculometer and Head Tracker 

As shown in Figure 3-1 calculating the point of gaze (POG) has become more complex 
than with a fixed oculometer head and additional sources of error have been introduced. In order 
to reduce overall system error, more care must be taken to assure that the head is initially placed 
in, and does not vary from, a known position during calibration. The subject’s head position must 
now be adjusted until two LED's, one on the right and one on the left of the cockpit, each become 
visible through a pm hole frame. Once the subject has been boresighted into the correct head 
position the helmet is restrained to assure minimal variation from that position during calibration. 
After the helmet is restrained, the oculometer calibration proceeds normally, although 
adjustments to visor position may be required for optimal discrimination. Upon completion of 
calibration, the helmet is unrestrained. CORS software can now calculate a point of gaze in the 
calibration plane no matter what head position is taken by the subject. In the subsections which 
follow, the CORS eye/head data integration algorithms are described in detail. A description of 
the notation is followed by a discussion of calibration, the transformation of EVM coordinates to 
real world coordinates and extrapolation into the viewing plane. 



Point of 
Gaze (POG) 


Figure 3-1 . Illustration of the relations of the algorithms to calculate real world 

position. 


Notation 

A vector is defined as magnitude along a specified direction. Vectors will be written as: 
R = <3,4,5>, 
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meaning vector R has a magnitude of 3 along the x-axis, magnitude 4 along the y-axis and a 
magnitude of 5 along the z-axis. Vector addition is defined as: 

R + O = <3 4,5> + <6,7,8> = <9,1 1 ,13> = RQ, 

where the components of R and O are added together to produce the resultant vector RQ . A 
vector can be used to describe the relationship between two points. A point is defined as: 

P = (2,4,6), 

and is used to define a specific location in Cartesian coordinates. The vector between two points 
is the difference of the Cartesian coordinates of the two points: 

P = (2,4,6) and T = (3.5,7) PT = <3-2,5-4,7-6> = <1.1 ,1>. 

The vector PT is constant under any orthogona 1 transformation of the coordinate axes. An 
orthogonal transformation is a rotation or translation of the coordinate axes of the system. In 
CORS vectors only undergo a rotational transformation. A rotational transformation is the 
movement of the coordinate axes through three angles, which are generally called euler angles. 
To apply a rotational transformation to a vector or a point, a 3 x 3 matrix must be used. A rotational 
transformation matrix is defined as: 



<x 

xy 

xz 

R = 


yy 

yz 


zx 

zy 

zz 


the values of components are calculated from a group of trigonometric identities. Vectors which 
have undergone an orthogonal transformation are designated with primes. 

P = R * P 

The combination of the vectors, rotational transform matrices and points are the crucial elements 
of the algorithms. 

All coordinate systems are based on a point of reference or origin. Three different 
coordinate systems are used in the algorithms, they are world, EVM, and the display. The point of 
reference for world coordinates is the head tracker source. The origin for the EVM coordinates is 
defined as C5, the center point of the calibration pattern (see Figure 4-1, in the next section), 
although this is not the origin for EVM output coordinates. The origin for the display is the lower 
left corner. 
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Calibration Procedure with Respect to the Algorithms 

The most important assumption made during the calibration procedure is that the vector 
connecting the eye to the center calibration point (C5) is parallel to the z axis of the real world 
coordinate system. What this means in practical terms is that the test subject's eye is at the same 
height as point C5 and is centered directly opposite to point C5. An additional requirement is that 
the calibration pattern is perpendicular to the z axis of the source. With respect to the algorithms 
the calibration procedure consists of forcing the subject to align with the solid geometry 
mathematics by adjustment to the boresight position and then actually calibrating the subject. 

Three things must be accomplished during the calibration procedure, vectors P and R 
must be established and the boresight command must be given to the head tracker. As shown in 
Figure 3-2, P is the vector between the eye and the point in the EVM plane at which the eye is 
gazing. (With the head restrained, the EVM plane falls in the same world position as the display 
plane.) The value of P corresponds to the line from the eye directly to point C5 when the subject 
is in the boresight position. After the head and the eye have been correctly positioned and the 
head restrained, the boresight command is given to the head tracker. This command causes the 
head tracker to set azimuth, elevation, and roll to zero for the test subject's orientation during 
calibration. The remaining action is the calculation of R. As shown in the figure, this is the vector 
connecting the Sensor (B) and the center point of the calibration pattern (C5). R is calculated on 
the basis of the head tracker output with the helmet locked in the boresight position. 



Figure 3-2. Calibration configuration vectors. 
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Transformation of EVM Coordinates to Real World Coordinates 


The transformation of EVM coordinates to real world coordinates begins with the EVM 
output. Since the EVM cannot be configured to output 0,0 for the center point of the calibration 
pattern, the EVM output must be put into the proper form; 

OC x = (PEVM X - C5 X ) * factorx and 
OCy = (PEVMy - C5y) * factory 


Factor is the multiplier to transform EVM coordinates into world coordinates. OC is defined as 
<OC x , OCy,0>, which must be added to R to produce ROC. These variables are identified in 
Figure 3.4. ROC must be orthogonally transformed using the quaternions provided by the head 
tracker. The correct orthogonal transform, Q, is defined as: 


Q 


qO 2 + q1 2 - q2 2 - q3 2 
2(q3q0 + q1q2) 
2(q1q3 - q0q2) 


2(q1q2 - q0q3) 
qO 2 - ql 2 +q2 2 -q3 2 
2(q1q0 + q3q2) 


2(q1q3 + q0q2) 
2(q2q3 - qlqO) 
qO 2 - ql 2 - q2 2 + q3 2 


Applying Q results in ROC', the results of all of this can be combined to produce the real world 
coordinates of P. 

POG x = ROC x + Bx 
POGy = ROC y + By 
POG z = ROC z + B z 

Figure 3-3 presents the complete series of computations which are performed to obtain 
the screen location of the subject's point of regard. Oculometer, calibration, and headtracker data 
are integrated in this way 60 times a second, allowing interaction between eye position and 
simulated task to occur in real time. 

Extrapolation of the Point of Gaze (POG) Into the Panels 

To extrapolate the POG into the plane of interest in the operational environment requires 
combining the POG with P and OC (see Figure 3-4). P is the vector from the subject's eye to 
point C5 and OC is the vector from point C5 to the EVM coordinate point in real world coordinates. 


POC = P+OC 
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Figure 3-3. Oculometer and Polhemus calculations for integration. 



POG and the POC represent ihe line of sight. The next step is to determine the closest plane to 
the POG. This is accomplished by calculating the distance between the POG and the centers of 
each of the planes. Once the closest plane has been established, the POG is extrapolated into 
that plane along the POC. The intersection point for that plane and POC is calculated. The 
coordinates of the intersection point are compared against the maximum and minimum values for 
each Cartesian coordinate. If the intersection point falls between all of the minimums and 
maximums then the real world point is transformed into coordinates corresponding to that plane. If 
the intersection point does not satisfy the minimum and maximum conditions, then it is necessary 
to check another plane. The next plane to be checked is determined from a search order that is 
part of the definition of the cockpit. The search order is a list of planes that should be checked if a 
POG is close to that plane, but does not actually intersect that plane. At the top of the search 
order are those planes which are the closest, since they are the most likely candidates to be the 
correct plane of intersection. 



Figure 3-4. Operational configuration vectors. 

After a correct intersection point has been found, it is transformed into the coordinates 
that define that plane. The region ID for that planar coordinate is determined and is the output 
from CORS along with the plane ID and the time that the fixation occurred. The only exception to 
this is if the region ID is an invalid region (invalid regions are those regions which are used to make 
an irregular plane a rectangle). Because an invalid region does not actually exist in the operational 
environment it is necessary to proceed in the search order to the next plane to determine the 
correct intersection. If the entire search order is exhausted without a valid intersection being 
found in any of the planes, then CORS outputs that a fixation occurred at an unknown location. 
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The Processing Components: The CORS Computer 

The CORS computer includes the Model 60 Compaq 386/20 MHz with VGA monitor, disk 
drives, disk controller cards and various special purpose cards: serial communications, parallel 
processor card, and 80387/20 math co-processor. A complete list of components is as follows: 

Compaq 386/20 MHz, Model 60MB Hard disk, 640K memory, 

1 MB RAM Upgrade, 

1 .2 MB 5 1/4 inch floppy drive, 

VGA monitor, 

Vega VGA board, 

80387/20 MHz Math Co-processor, 

Applied Reasoning 386/16 MHz Parallel Board, 1 MB memory, 

Applied Reasoning Developer Kit, and 
DigiBoard Com/xi, 52K bps maximum. 

Description and Purpose of Component Cards 

The Compaq 386/20 MHz computer has 8 slots for cards. Five of these are full height 16 
bit slots. Two are 8 bit slots, one half height and one full height. Finally, the remaining one is a 32 
bit slot. 


The Compaq motherboard acts as the controlling processor. It is responsible for memory 
transfers between the various cards. 

The parallel processor board, which has a 80386 processor with a 1 6 MHz clock, resides 
in a 1 6 bit size slot. This board handles the algorithmic processing for CORS. It expects incoming 
decoded data to be placed in ring buffers setup in a megabyte of memory. The data are the 4 
quaternions, 3 Cartesian coordinates, the x and y position of the eye, and pupil diameter. When a 
new fixation point has been established, it outputs these data to another ring buffer in its memory. 
These data consist of a number representing a region. The Compaq motherboard has the task of 
retrieving the data from the parallel processor board memory and sending the data to the FDAU-M. 

The Com/xi serial communications card, which also resides in a 16 bit slot, is used to 
accept data from the head tracker and the EVM. It supports four serial ports, each capable of 
being configured separately. The Com/xi is a smart communications card with an onboard 
processor and it performs communications independently of the Compaq motherboard. One of 
its features is that it can be configured to ring buffer the input automatically. There is 8K of dual- 
port RAM available for buffering, half of which can be used for any given port. 
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In the case ol communication with the EVM, the Com/xi ring butters eight byte records 
every 1/SOth ol a second al 9600 bps. The motheiboard letches EVM records, one at a time, 
tom the Com/xi and transfers them to one of the parallel processor board input nng buffers in the 
parallel board memory. Before transferring the data, the processor on the motheiboard decodes 
and verifies that the check sum word indicates that transmission was error free. 

in the case of communication with the head tracker, the Com/xi ring buffers 20 byte 
records every 1/60th of a second at 19.2K bps. The motherboard fetches head tracker records, 
one at a time, from the Convxi and transfers them to one ol the parallel processor boants input nng 
bullets in the parallel board memory. The motherboard processor first has to decode the ea 
tracker records and reverse the bytes in the seven words before transfemng the data. 

The two remaining 1 6 bit slots are used for the hard disk, floppy disk, and two pods; one 
parallel and one sehal. One of the 8 bit slots, the full height slot, is used for the Vega VGA board. 
Finally, the 32 bit slot is used for memory expansion to the maximum ot 640K of memory 
addressable by the CORS computer which is more than sufficient tor the CORS application. 


Runtime Software 

The architecture ot the runtime software is shown in Figure 3-5. These functional pieces 
are shown in relation to the three main hardware components, serial board, motherboard a 
parallel processor board. The algorithms discussed above for integration of the eye track and 
head track signals are shown under the parallel processor. The spatial and temporal filters used in 
processing will be discussed below. 

The software was first developed on the Masscomp and then transpoded to the CORS 
computer. This provided ihe dual advantage ot not only developing the required CORS software 
before the CORS computer was available but also provided a separate testbed for evaluating 
various components of the system. This testbed was utilized in some of the experiments 
described in Task 2. User manuals tor the runtime software are provided in the Supplemental 

Volume, Part A. 


CORS Input 


The input to the CORS algorithms consists of Eye 
tracker tracking data. Data from both of these sources are 
Theoretically, the data could be communicated in serial 


View Monitor (EVM) data and head 
communicated sixty times a second, 
or parallel fashion. When reading 
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Figure 3-5. Architecture of the runtime software. 















data using the motherboard processor, parallel communication is usually preferred because it is 
faster. Since we cannot afford to burden the motherboard processor, it is essential to have a card 
perform the communication independently of the motherboard processor. When the hardware 
for this project was selected, there was not a card available that performed parallel communication 
independently from the motherboard processor; however, a serial card of this type was available. 
This independence is preferable so that the motherboard processor is available to do other time 
critical tasks. Consequently, the EVM and head tracker communication is done serially in CORS. 

Communication with the EVM is done at 9600 bps. The algorithms require three words 
(two bytes per word) of information from the EVM every 1 /60th of a second: x position of the eye, 
y position of the eye, and pupil diameter. Every record that the EVM sends ends with two check 
sum bytes (check sum word). The check sum word is the sum of the vertical position, horizontal 
position, and the pupil diameter of the eye. The record transmitted by the EVM is eight bytes (3*2 
+ 2 = 8 bytes). The check sum bytes are used to identify the boundary between EVM records and 
to perform error checking on the transmission of the data. Because the words transmitted by the 
EVM are transmitted a byte at a time, the mother board must perform a decoding process on the 
EVM input data as well. In this case, the decoding is simple: All that is needed is to put the bytes 
back together again as words. 

Communication with the head tracker is done at 19.2K bps. The algorithms require seven 
words (2 bytes per word) of information from the head tracker every 1/60th of a second ;hree 
Cartesian coordinates and four quaternions. Every record that the head tracker sends begins with 
three status bytes. The 17 byte record (7*2 + 3 = 17 bytes) is transmitted as 20 bytes because it is 
in binary encoded format. The most significant bit (MSB) for each seven data bytes are stored in 
overflow bytes. This encoding makes the MSB in every byte 0, except for the first byte in each 
record which has a 1 as the MSB. This is done to make it easy to determine the beginning of a 
head tracker record. In addition, the bytes in words are transmitted backwards, the low order byte 
followed by the high order byte. The serial board transfers the input data to the mother board 
memory, the mother board must decode the data before using them. 

Explanation of Implementation of CORS Algorithms 

The objective of the CORS algorithms is to identify those segments of the positional data 
(eye and head) which represent the operators points of attention. In order to do this, motion 
between dwells must be filtered out. In the CORS algorithms, this is accomplished by means of 
two main loops; the motion loop and the fixation loop. The motion loop is executed whenever the 
test subject is in transition between points of attention. The fixation loop is executed while the 
test subject is attending to a specific object in the visual field. Figure 3-6 demonstrates the flow of 
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Figure 3-6. Flow chart of the point of gaze (POG)/fixation determination. 
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these two loops and their relationship. The lowest row of operations in the fixation loop represent 
that part of the loop which can be omitted when it is not necessary to determine when the test 
subject stops looking at an object or when the user is only interested in dwells and it is not 
necessary to identify gaps between successive fixations. 

Spatlotemporal Filtering 

The motion loop (Figure 3-6) is the processing loop done most often. This loop uses a 
new single time frame of information, a head tracker record and an Eye View Monitor (EVM) 
record, and checks to see if a point of gaze (POG) exists. Testing for a POG involves averaging 
the most recent twelve lookpoints in the EVM imaginary plane and comparing these twelve 
against the average. If ten or more points fall within a specified radial distance from the average, a 
POG has been established in the imaginary plane. Where the test subject is actually looking in the 
world has not yet been determined, only that the test subject is looking at some location and is not 
currently in transition. If a POG does not exist, the motion loop starts over again. If a POG does 
exist, the fixation loop takes control. The operation of the spatial component of the 
spatiotemporal filter is illustrated in Figure 3-7. In each panel, the circle represents the spatial 
portion of the filter while the dots indicate 12 successive lookpoints in the EVM plane. In the 
figure, only the leftmost panel shows data that qualify as a POG. For the other two, there is not a 
sufficient number of points within the circle to qualify. The radius used was 0.316 inches or 52 
minutes of arc at the calibration plane. 



Figure 3-7. Illustration of the spatiotemporal filter operation. The radius is 52 minutes 

of arc. 

A note on the filters. Twelve time frames correspond to 200 msec. There were several 
reasons for using 10 out of 12 successive frames as the temporal portion of the filter. First, there 
was some uncertainty with regard to the speed of the CORS computer and whether lookpoint 
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frames could be processed fast enough in real time. Second, the DFDR writes a 12 bit word in 
15.675 msec. For prototype purposes, a number of extra words are written to the DFDR. The 
CORS computer provided four words and four are generated independently within the FDAU-M. 
These include time codes from the CORS computer, dwell regions, etc. as well as extra 
independent time codes generated within the FDAU-M. These FDAU-M generated codes are 
available to be used for validation. The total time necessary to write eight 12 bit words on the 
DFDR is 125 msec. Thus, during prototype development, the decision was made to ignore 
fixations of durations shorter than 200 msec e.g., 'glances' -100 msec, to make it possible to write 
the additional information and also to avoid overloading the DFDR and the CORS computer. The 
time can be defended because it will capture a great percentage (>90%) of instrument 'reads' 
(Harris et al. 1986; Harris & Christhilf, 1980). Further, because the number of words written to the 
DFDR can be reduced from the prototype version, this does not represent a serious or permanent 
limitation for the CORS system. 

The first thing done in the fixation loop is to process the POG in the imaginary plane to 
obtain a real world region in a plane of interest. This region is called the fixation. Using the 
fixation, a region identification number is looked up. Most of the time, the region ID number will 
correspond to an instrument in the plane. Alternately, when the dwell is a region in a plane that 
does not contain an instrument, it is called the background region. 

If the previous loop was also a fixation loop, it is possible that the test subject was already 
looking at the current region. A test is made to check if the region ID number corresponds to a 
new region. If there is a new region, the region ID is output to the FDAU-M. 

The next stage of the fixation loop is to read/decode records until a POG no longer exists. 
The purpose of this is to determine when the test subject stopped looking at a particular region or 
object. Once a POG no longer exists, a different POG cannot possibly exist for another ten 
frames including the lookpoint that caused the termination of the previous POG determination. 
Thus, nine records are read and decoded without any POG processing. Then, the most recent 
twelve frames are processed for a POG. If a POG exists, the fixation loop continues. If not, the 
motion loop takes control again. 

The calculation of a fixation point in world coordinates from a POG in EVM coordinates is 
the heaviest single processing demand imposed by the CORS algorithms. For this reason, entry 
to the fixation loop occurs only when absolutely necessary. Additionally, the fixation loop can 
never be exited and reentered without first processing through at least nine lookpoints. 

Determining POG Existence 

To determine whether a POG exists or not, twelve time samples are used in the moving 
window. The twelve calculated points in the EVM imaginary plane are compared against their 
average. (The EVM imaginary plane is located at a fixed distance from the eye and is in a fived 
position relative to the head.) If a predefined number of the lookpoints fall within a specified radial 
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distance Item the centroid (average), existence ot a POG is established. For prototype purposes, 
jen'ts being used as the initial value o. the minimum number ot lookpoints that must be ,n range 
in the l, oures that follow a time line demonstrates the twelve trame windowing process used 
determine existence ot a POG. The time line scaling is 60Hz because data are input from both the 
*™r atdThe EVM sidy times a second. The biack bars wkhin a window represent valid, m 
range lookpoints. The white gaps within a window represent invalid, out ot range lookpo.n s. 

Temporal Processing: An Example 

Figure 3.8 demonstrates a possible case that could exist before a fixation ^ started 
this case, the tweive points were averaged and oniy nine of the tweive were found to be ,n 

range. . , 

actual end 

actual start of fixation 



9 samples within 
radial distance 


Figure 3-8. An early phase of fixation determination. 


Figure 3-9 shows a possible outcome two 
additional points changed the centroid (average) 


lookpoint frames later than Figure 3-8. The two 
to that of ten out of the twelve points in range. 


actual end 

actual start of fixation 

of fixation PO(S established 



10 samples within 
radial distance 


Figure 3-9. Fixation established. 
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Therefore, a POG exists and a fixation point can be calculated, 
established until twelve frames after the fixation actually started. 


Note that the POG is not 


fixalinn in ,h Processing conlinuing to take place. The test subject Is still 

t ng ,n same region. In this panlcular time window, ten out of the twelve points are in range 
is is representative ot the stage of the llxation loop that continually loops until a POG no longer 

C Alois* ^ 


actual start 

of fixation POG established 


actual end 
of fixation 



10 samples within 
radial distance 


Figure 3-10. Analysis continued with fixation being maintained. 

Figure 3-1 1 demonstrates the termination of a fixation. Only nine of the twelve points are 
-n range. Note that it is not realized that the POG is lost until one or two frames after the fixation 
ac ua ly ended. In the fixat.on loop, this corresponds to the test for the termination of a POG 
bemg true ' At ,hls P° in <. nine records can be read and decoded without doing any POG 
processing. A new POG cannot possibly exist until ten frames after the fixation terminated 


actual start 

of fixation POG established actual and 

I . of fixation POG 





9 samples within 
radial distance 



Figure 3-11. Fixation lost, point of gaze changed to another position. 
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CORS Output 


as lollows: 

Panel (8 bits. 1 byte) - a number representing a panel area m the cockp • 

a: in a nanfi in 


1. 

2 . 

3. 

4. 


panel vo uu=>, < • . . ... 

Region ID (8 bits, t byte) - a number representing a region in a panel in 

cockpit. . „ 

Dwell Start Time (24 bits. 3 bytes) -the transmission time ol the dwell. 

Synchronization Time (6 bits. 1 byte) - the difterence between data arnval 
and dwell start transmission. 


a i »\ji vi**'-*" * * 

There are three types o, internal encoded in the ^ 

Stan o. fixation or dwell, location o. fixation or dwell, and ^ or , nu# 

ID, indicating a dwell on a region out contain a value simply Indicating 

a , ermine, ion-o. record is transmhted, the ^“" “^'^ “ erminat^n o, intormation 
termination, with no 

In an operational system, on, the two bytes represented by 
points 1 and 2 above are needed, specifically, the panel and region. 


Relation of Fixations and Dwells: An Example 


Fixations and POGs refer to a specific spatial point in dW " 

more fixations (with or without i^^^^^ S ^gfJi° areas whiC h contain information and 

specific region. An ins rume eas within the instrument boundaries (Harns 

require separate eye movements to these sep computer to the DFDR are coded in 

et al., 1986). Accordingly, the data sent out of the CORS computer 

terms of points and regions. 

Data compression occurs by convening the 

fixations and dwells. Ye, another level o, 

optionally, either me begmning and end * -he tfcure, 

Figure 3-12 Illustrates the relation ol lixations g h 9 six fixations in the 

mere are tour dwells which would result in tour records. By contrast, there 

figure which would result in 12 fixation records. 
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Figure 3-12. Illustration of the conversion of fixations into dwells. 

Other Features of the Output 

CORS also writes a start-of-run record and an end-of-nin record. Byte 1 represents tlao 
Ms (11111110 lor Stan and Ittltin tor end) and is written to the FDAU-M. The remaining 
ytes in the record are used tor the experiment name (in ASCII) written to both FDAU-M and tile. 

M The FDA R u man 0U,PU, L eC °' dS * C ° rnmUniCaled "'Parallel tashion to the FDAU- 
9 reCWds " receives w,lh lime code generator (TCG) time. This new record is 

D ~: '° be S, ° red Laler pos, - run anal ^ fe <*" made by synchronizing the 

the TCot^el Tr Wa reCOrd ' nS ' hr0U9h and 

It was possible that the TCG clock and the Compaq dock did not measure time at precisely 
the same rate. The purpose ot the synchronization lime m the record sent to the DFDR is to 
cone, ate (upon retrieval trom the DFOR, time tagging in the FDAU-M with the time taggi™ £ 

and^he TCG 0 ^ 5 * 8 reSU "' n ° PhVS ' Ca ' " ' S re9uire ‘' CORS compute, 


The Recording Components 


The Flight Data Acquisition Unit Emulator (FDAU-M) 

The objective of CORS is to write eye scan data on the DFDR. However the CORS 
computer and the DFDR employ ditteren, communication standards. Accordingly the FDAU-M 
“ ~ 7 necessar y interiace between the CORS confer ajhe d f d r T h “ 
took the place of a much more expensive, commercially available, FDAU The FDAU-M 
provides (bidirectional) data conversion between the CORS computer and the DFDR standards. 

Additionally as discussed above, a provision was made to incorporate a TCG signal to 
allow independent time recoids to be established on the DFDR. The TCG signal was a vaLion 
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,o be compared against the time codes provided by the CORS computer. Further, the ICQ 
signals were recorded on the real-time video made through the scene camera. 

The high level design and interrelations are shown in Figure 3-13. This figure shows the 
detailed functional design ol the FDAU-M represented by the dotted lines with the CORS 
computer, the sensor components, the TCG. and the DFDR also depicted, outsrde the dotted 

lines. 

Theory of Operation 

The theory behind the FDAU-M operation is simple; it is a pure hardware/firmware device 
that performs data format conversion. 

Data enter the FDAU-M using RS-232 data format via a universal synchronous/ 
asynchronous receiver/transmitter (USART). The USART (Intel 8251 A) is designed to handle 
bidirectional communication with the CORS compter. The CORS computer data are input into 
Ihe FDAU-M in the form ol 8 bit words at any (selectable) RS-232 baud rate. TCG data are input 
into a butter in parallel format. The correct bits are masked out and arranged. CORS computer 
data are grouped in bursts that represent dwells. These dwells are then time tagged. All data are 
men converted into 12 bit format, maintaining data integrity. The newly 'or^d '2 bit words are 
placed onto a tirst-in. first-out (FIFO) 12 word butter. The data are then shrlted out ot the butter 
serially one bit at a time in Harvard biphase format at 768 bps. Sixty-tour 12 bit words const, tu e a 
subframe. The first word nl each subframe is a special identifier (stored in ROM). T ^ re ar ® 
subframes in each frame. After the firs, subframe marker, the frame counter word is added There 
is also a null word when the data buffer is empty. All words contain special idem, Her bits used 

decoding during playback. 

DFDR Playback 

The FDAU-M also retrieves data from the DFDR and returns them back to the computer. 
This is done by using a read amplifier board and a bit synchronization card trom a read data unrL 
The data are retrieved from the DFDR via six unconditioned analog signals recovered rom 
playback magnetic head. The read amplifier board and bit synchronization cards are then used to 
titter and condition the signal to a digital (NRZ-L) formal and a corresponding clock. These ca d 
provide considerable latitude in allowing the designer to control signal gam in vanous places whm 
allow noise to be controlled. Also, the output signal choice can be made which increases usa it V 
under various conditions. These boards also provide provisions lor track selection which ,s usetul 
during the playback phase. Both cards were furnished by Fairchild Weston. The data are hen 
recovered trom the cards in NRZ-L data format. The FDAU-M then converts the data to selectable 
RS-232 format and outputs the data to the computer. 
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Figure 3-13. Relations between the FDAU-M and other 


components of CORS. 















The Digital Flight Data Recorder 


The DFDR used for the CORS system is Model F800 from Fairchild Weston. It is a six track 
continuous loop magnetic tape device that can withstand 20.000 lbs of force per axis, impacts of 
1000 g's per axis for 5 msec and can withstand temperatures of 1100° Celsius. It records at a tape 
speed of 0.36 inches per second and can record continuously for 25 hours (four hours and ten 
minutes per track without overwrite). The DFDR accepts ARINC 542 or 573/717 electrical 
standards. The ARINC 573/717 standard was selected for CORS. It uses 768 bps, Harvard 
biphase, 12 bit word synchronous serial protocol. 

The DFDR can be configured, through FDAU-M control, to operate is such a manner that 
data can be read three seconds alter they are recorded or the data can be read off at a later time as 
would be done in ground playback of normal DFDR data. 


The Time Code Generator (TCG) 

The TCG used is model number 9100A built by Datum, Inc. The TCG is a setable time 
keeping device. The device is set by entering the desired time using a thumbwheel 
switch/momentary switch combination. By pressing the start button the TCG starts keeping time, 
as shown on the front LED display. The TCG maintains the Julian day, the military hour, minute, 
and second. The TCG also maintains time up to 1 ms, but this is not shown on the front display. 

Standard outputs include IRIG B (resolution (1 sec.) and 180 degree pulse trains ranging 
from 1 PPS to IK PPS). The optional outputs that were used include a 50 bit positive logic 
parallel signal (resolution, 1 ms) and a video time insertion signal compatible with NTSC signal 
standards. This mode was used to put a time code on the recording of the scene camera output. 


The Cockpit 

A 'wire' frame was constructed to serve as a simulated cockpit. Figure 3-14 illustrates a 
schematic view of the cockpit; Figure 3-15 shows a perspective view, and figure 3-16 shows an 
unfolded view.. The design goal was to attempt to replicate the panel positions a commercial 
cockpit as cost effectively as possible while maintaining single pilot occupancy. 

The frame was constructed of PCV (1.5" diameter) and serves as support for plastic 
panels which were attached by means of tie wraps. This approach allows for the rapid 
replacement/substitution of panels that may be required for different experimental purposes. 
These panels provided a flat surface upon which instrument facsimiles could be placed. The flat 
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surface is of considerable convenience in establishing the dwell regions (setup) for CORS. The 
frame also supports the receiver of the head tracker. 



Figure 3-1 4. Side and cross section views of the cockpit. The left panel is a 
side view, nose to the left. The other two panels represent cross sections 
taken at the points indicated in the left panel, a corresponds to the 
lookpoint, b corresponds to the eyepoint. (The scale indicates feet.) 



Figure 3-15. A solid perspective view of the cockpit, nose to the right. The panels 
are numbered to correspond with Figure 3-16, where 3 and 8 represent the front 
(visible), 16 the back (to the left out of view), and 17 represents the floor 
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Boresights were installed to provide references points to establish the head in a known 
position daring calibration. A boresight consisted ol a light mounted behind two p.nholes about 
nine inches apart. In order to see the light, the eye has to be positioned directly in line with the 
pinholes. Two such boresights, located approximately 40- on either side ol the m,dl.ne provided 

fairly precise positioning of the eye (and head). 



Ftaure 3-1 6 Cockpit panel configuration. In this unfolded view, the test subject 
be slated on area 17, panel 8 would be a. eye^e. In * 
behind him. Panel numbering is the same as in Figure 3-15. (Scale is in feet.) 


System Software 

Three system software products were developed. The first and second ot these were 
designed to handle setup which involves identilication ol cockpit regions and planes lor the 
CORS system. The data provided by these programs are then used by the CORS computer 
during runtime processing. The third piece ot software was developed for playback and analysis 
of the CORS data once they have been recorded. 
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The Planes Designer Program 


The planes designer program permits the user to enter coordinates of the planes and 
regions within planes so that conversion to dwell regions is possible. For example, the planes in 
the cockpit illustrated in Figure 3-16 are entered to permit CORS to relate the integrated eye-head 
position data into panel areas. A few of the highlights of the program are described below. (The 
user manual is provided in the supplemental volume, complete with an example of a plane being 


All panels used in the planes program must be represented within a 
rectangle. 

All panels must be located in a global positioning system. This positioning 
system is designated as world coordinates and is based on the head tracker 
un J®- The head tracker source origin is considered the origin of the 
CORS world coordinate system. All planes must be referenced with regard to 
the head tracker source unit. y 

The normal for a plane is the vector that originates from the center of the 
plane and is perpendicular with respect to the plane. This vector must be 
given in world coordinates and normalized to a unit vector. 

The planes designer program needs the minimum and maximum X Y and Z 
values that occur in each plane. ' ’ 


The Setup Program 


A cockpit is composed of a number of planes or panels. The planes designer program 
discussed above is used to create a separate file for each plane in the cockpit. The execution of 
both runtime and playback programs require that the user specify a setup file. This file is used to 
determine which plane files to read and to establish the search priority for each. The setup 
program provides this cockpit specification for CORS as an integrated set of planes. 

The program asks the user for the number of planes in the cockpit, the names of the 
panel files which represents each panel, and then the search order for each panel. An ascii text 
file is written out by the program in the following form: 


number of planes 
panel name 1 


panel name n 
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search order 1 


search order n 


Replay Software: Function and Application 

The CORS replay program allows the user to examine, via a graphical interface, the hies 
dumped from the CORS runtime or FDAU-M reader programs. Various menus are presented lo 
the user at start-up and dunng replay in order to control the presentation. Based on the setup or 
cockpit file referenced, the program reads in the panel tiles and translorms ihe cell matnx data of 
each into line drawing specitications lor that panel and Ihe regions which it contains. Dunng a 
replay run, the program presents one ol two display formats to the user. In the first forma, the 
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display presents a single graphical window, containing the outline drawing of a single panel and its 
regions, with text material below. In the second, shown in Figure 3-17, the graphical area is 
broken into four windows with the same text material below. Each of these windows may contain 
the outline of a single panel and its regions. In both formats, the graphical windows are labeled 
with the name of the panel files used to construct the images. The text material displayed below 
the graphical window(s) is the same regardless of the display format. 

In the bottom left of the display, four items are shown which remain unchanged 
throughout a replay run. These include the date and time of the original run, the device utilized 
(head tracker or helmet mounted oculometer) and the type of file (fixation or dwell, see page 26) 
upon which the replay is based. In the bottom right of the display two times are shown. The lower 
of these times represents the current (replay) time while the upper indicates the start time for the 
fixation or dwell currently presented in the graphical portion of the display. Figure 3-18 presents 
the appearance of the display during a replay run. The current time indicates that the user is 


Left Panel 


Right Panel 


Overhead Panel 



Center Console 


Friday August 11 1989 16:15:19 

DWELL HELMET MOUNTED OCULOMETER 


00:11:56:12 

00:11:56:89 


Figure 3-18. Illustration of an example in the display windows. 
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eleven minutes, fifty-six and eighty-nine one-hundredth seconds into the replay. The dwell start 
time indicates that the shaded region shown in window three represents a dwell which began 
seventy-seven one-hundredths of a second before the current time shown. 

As a replay progresses, successive fixations or dwells are indicated by changing the color 
of regions' outline and fill. If replay encounters a fixation or dwell in a panel not currently displayed, 
a currently displayed panel is erased and the required panel drawn in its place. In the four-panel 
display format, window 4 is the window used for this type of display update. Windows 1 through 3 
retain their original panel assignments throughout the run. 

Replay can be paced by the user or by the clock. In the case of user pacing, the program 
steps from one fixation or dwell to the next, in sequence, in response to keyboard input. In the 
case of clock or automatic pacing, the program attempts to run in real time. That is, if a dwell was 
three seconds in length at run time, it should be three seconds in length during replay. However, 
the drawing speed of the Compaq is such that delays are inserted whenever a drawing operation 

is required. No attempt is made by the program to compensate for these delays. The program is 
running real time when the playback time readout is seen to be advancing. 

The replay program assumes that the setup file is a local file named SCREENS.DAT and 
that the replay file is in a local file named TIME.DAT. In order for the replay to execute properly, it is 
necessary to provide as local files those panel definition files used during the original run. When 
executed, the replay program immediately presents the user with an assignment menu. A list of all 
the planes specified for the run is displayed with a space alongside each for the entry of a window 
number. Function, delete, and up and down arrow keys are used to specify window assignments. 
If an assignment is made for window 1 and no other, the single window mode is automatically 
selected. If more than one assignment is made, or if a single window other than window 1 is 
assigned, the four window mode is selected. 

The function key F5 is used to indicate that the window assignment is complete. While 
the program loads the required panel definition files, a version message is displayed. When the 
loading operation is complete, the windows are drawn and a start-up menu is displayed. At this 
point the user may choose either automatic (clock) pacing or stepped (key driven) pacing. Once 
replay has begun, it will proceed until the end of the replay file is encountered or until the user 
depresses the escape key. The escape key brings up the control menu, a list of functions with 
corresponding function keys, giving access to the following operations: 

• Change replay pacing - if replay is currently being stepped manually the Auto 
Step alternative will be shown; if replay is currently clock driven, the Manual 
Step alternative will be shown. 

• Select start time - fast forward to specific point in replay file (not currently 
implemented). 

• Reassign windows - program presents same window assignment menu used 
at start of program. 
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• Select event type - set program to replay dwells, the default, or fixations 
(fixation replay is not currently implemented). 

• View algorithm parameters - displays values of all CORS algorithm parameters 
(number of points for fixation window, maximum number of invalid points in 
window, square of radius, number of frames skipped, drift limit) as they were 
set during run being replayed. 

• Exit program - terminate replay run. 


Step 1 

SETUP 




Step 2 

SENSING 


PROCESSING — ► 

RECORDING 


Step 3 




PLAYBACK 



Figure 3-19. Simple diagram showing the relation of the steps and the operations 
performed within each step of CORS. 


System Summary 

Overall the prototype represents a workable and flexibly designed CORS system. For the 
most part, the hardware used for the prototype system is off-the-shelf. The single exception is 
the FDAU-M which was designed, developed, and built specifically for this effort. All of the 
hardware items are employed in Step 2 of Figure 3-19 and include: 

• The sensing components 

- A helmet mounted oculometer (HMO) with a video recordinq feature 
and 

- A head tracker magnetic head position sensor. 

• The processing component 

A personal computer with a parallel processing and a serial board 
added. 

• The recording components 

- A specially built flight data acquisition unit emulator (FDAU-M) and 

- A commercial aircraft digital flight data recorder (DFDR). 

• Miscellaneous items 

A simulated cockpit and 
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- A time code generator used to provide timing signals for the video 
recording and purposes of validation. 


All of the software was developed specifically for this effort. This includes the following 

pieces: 

• The setup software which permits the user to enter the cockpit configuration 
and to identify dwell regions for use by the CORS system, during both 
processing and playback. 

• Three conceptually different functions are performed within the CORS 
computer during data collection. The software functions are: 

The sensing software which integrates the 60Hz data from the 
helmet mounted oculometer and the head sensing units. 

- The processing software which uses the newly formed position 
signals, applies a spatiotemporal filter and determines fixation/dwell 
position. 

The recording software encodes the results and transmits them to 
the recording medium (screen, disk file, or flight data recorder). 

• The playback software which allows the user to retrieve the recorded data and 
examine them in the context of the cockpit regions identified with the setup 
routine. 
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TASK 2: CONDUCT EXPERIMENTAL STUDIES 


The overall goal In Task 2 Is: To conduct experimental studies In 
the CORS laboratory. Experiments will be performed initially to 
verify system operability and to provide design feedback. Later 
experiments will be performed to refine system parameters, 
develop optimal techniques, and to verify the utility of CORS In re- 
constructing pilot eye movement history. 


The Testbed 


Because of the graphics available on the Masscomp computer, some of the experimental 
work was done using the Masscomp as a testbed. In these cases, the EVM and the head tracker 
were connected to the input ports of the Masscomp instead of the CORS computer. The helmet 
mounted EVM was used in all cases. The CORS software and algorithms were utilized whether 
the work was done on the Masscomp or the CORS computer. 

It is also of importance to note that the helmet was always restrained during the Masscomp 
testbed experiments, effectively eliminating the head tracker sensor inputs from consideration. 


Sources of Error and Characterization of Accuracy 


Generally, the error for most mechanical systems is directly related to how accurately the 
system can perform its intended function. CORS has three major components that contribute to 
the overall system error. Those components are the hardware, the data processing algorithms 
and the integration scheme of the system. Hardware refers to the oculometer, the head tracker 
and the computer which integrates the data. The data processing algorithms are the 
methodology used in the EVM to arrive at a location on the scene camera display where the eye is 
pointing. Additionally, data processing algorithms are used by the head tracker to calculate the 
translational and rotational movement of the head tracker sensor which is attached to the back of 
the oculometer helmet. The integration of the system components is the combining of the output 
of the helmet mounted oculometer and the head tracker to produce the POG. It is important to 
remember that the POG must be processed to determine a dwell, which is the combination of a 
series of lookpoints. An error value can be associated with a POG by evaluating the components 
described above, but the error associated with the dwell is a combination of the error from the 
POGs plus an evaluation of the effects of human physiology and saccadic movements which 
make up a POG. 
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Hardware and Software Components 


There are three hardware components to the CORS system. The first is the oculometer. 

n< hardware error in the oculometer, the pupil camera and the scene 
There are two sour pixels/inch leading to a positional error ± 0.009 

teading to a P 08 **"*' ^ accuracy „ , , nch RMS and an 

hardware compone* ^ thi , d hardware ^nont Is the Masscomp display; 

;r;~ai o ,:jed 0 n « * • ■— « - 59 

leading to a positional error of ± 0.017 inches. 

rdr^rr;^— 

Untortunale^Jtti^ ^^'^e^culo meter are those used to discriminate the pupil and 
processing algorithms u..ed in d ata 0 nto points taken during 

-r:::c=:rr: -f — ,hese 

algorithms and the human test subiect is the primary source ct error ,n the enure sys 

The error assoc, ated with integrating the output from the head tracker and the EVM 

— - — * r„rr^ra::r rrr:t ::: . 

rrstTutctThTadTs immobilized and there is c^y one ITmTa^Z 

rrrrrxr rr ::r «. . re, 

location as described in the Task 1 discussion. To 

rr;=rr=r=:rr,h ro ughom,heem, re 

series of calculations and induces an error in the final result. 


Averaging and Transformation 


The saccadic and mic-osaccadic movement ot the eye causes the line ot sight to fluctuate 
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actual fixation can be determined. While this is not intrinsically a source of error, the process by 
which the POGs are transformed into a fixation and the way that these fixations are then evaluated 
can affect overall system performance. 

Work done under this and previous contracts has led to a variety of filters and processing 
algorithms to transform raw POGs into dwells. Research has shown the filters and processing 
algorithms to be application specific. For example, instrument scanning results are influenced by 
the task to be performed (e.g., Dick, 1980). In the piloting task, the periods of fixation/dwells vary 
over a considerable range (approximately 60 to 600 msec [e.g., Harris et al., 1986]). By way of 
contrast, in a different task such as reading the range of fixation values is in the range of 200-300 
msec. Once an application has been specified, then the effect of utilizing an individual filter can 
be quantified and folded into the system error calculation. Filters can also be used to aid in 
compensating for system errors in the processing of the data. 

Fluctuations In System Performance: Informal Observations 

The goals for any operational system are reliable, consistent, and accurate performance. 
A reliable system does not require frequent maintenance and the system components are 
sufficiently rugged to be used in an operational environment. Consistent performance entails 
having the system produce the same results every time that the system is used. Finally, the 
system should be as accurate as possible within the limitations of its individual components. 

One critical technical issue that needs to be addressed in a future effort is calibration of 
the human user. Invariably, a test subject must be calibrated more than once before the system 
has an accurate mapping of POG onto the display system. Generally, one or more of the 
calibration points are not correctly aligned with the calculated display points. This indicates a 
deficiency in the calibration procedure, either in execution or internally in the oculometer 
processor or, alternately, in the test subject failing to look directly at the calibration point. There 
has been no observed correlation between various "bad" calibrations, i.e., no single source has 
been identified as a primary variable. Nor is it possible to isolate the source of error because the 
calibration routines and calibration data cannot be examined separately in this oculometer system. 

Another inconsistency in performance is the random fluctuations in the calculated 
lookpoint produced by the oculometer. Observation during testing suggests that they are 
caused by the inability of the oculometer to discriminate the pupil. Since the pupil and the cornea 
can both be observed in the pupil camera image, it is likely the discrimination algorithms have 
difficulty dealing with fluctuations in image contrast. 

The final problem dealing with system consistency relates to degradation in system 
performance over time. This seems to be related to test subject visual fatigue which causes 
differences in the pupil and corneal data as the experiment continues. The experiment described 
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in the next subsection was used to validate these observations of system performance and to 
provide data on the variations across calibrations. 

A primary goal ol this effort vras to quantity system accuracy. As was implied above, two 
levels ot accuracy need to be addressed. The first is the size ol the cluster ot lookpoinls that 
make up a fixation and the second is how tar the calculated fixation dilfers from the actual port of 
interest The size ot the cluster tends to be a function ol the human test subiect and represents 
the human limit imposed on system accuracy. The second is a system issue and can be quantified 
based on the algorithms used to calibrate the test subject and to interpolate the incoming raw 
data. The performance experiment described below was designed to provide intormation on both 

of these issues. 


Initial Experimentation 

The goals tor the experiment were to characterize the overall performance tor tixation 
determination; to characterize the changes across calibrations, across test subjects, and across 
the field-ot-view (FOV); and to relate present system deficiencies to individual system 
components. These goals were used to define a broad, simple experiment to provide preliminary 
data on the overall system performance. This was a preliminary experiment designed to pinpoint 
areas that needed to be explored more thoroughly. The puipose tor examining these factors is to 
identify those system components which require additional development and to develop an 
approach to improve the performance of those components. 

The first goal is to characterize the stability and quality of fixations as assessed by CORS. 
All static tasks and a high proportion of dynamic tasks require stable fixations. A stable fixatio 
occurs for example, when an operator identifies a point or region of a display associated with a 
control action to be taken (i.e., change airspeed). As previously described, the lookpo.nts must 
be processed to produce a POG which is then correlated to the desired point. The .ssue » to 
determine what is the size of the cluster of lookpoinls which represent the POG and the 
displacement of the POG from the point of interest. Further, because the clustering w.ll be related 
to calibration, an effort was made to characterize the changes that occur from one calibration to the 

next. 

One of the effects that has been observed during the experimentation conducted in this 
and in previous efforts has been that system performance degrades in the lower part of the field of 
view (FOV). Consistent problems in the lower regions of the FOV are related to the optics used to 
capture the pupil and cornea image and the discrimination algorithms which recognize the pupil 
and cornea and then calculate their centroids. 

A final goal in this experiment was to identify those components in need of improvement. 
For example, informal observations have suggested that problems exist in the calibration routines. 
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An objective was to have numeric data to justify the conclusions drawn from the informal 
observations. Once a more formal connection has been made between system components and 
poor performance, an approach can be constructed to improve the performance of those faulty 
components. 

Procedure 

The approach in this experiment was to examine stable fixations across test subjects. The 
Masscomp monitor was used to display the screen shown in Figure 4-1 . The test subject was 
calibrated using points Cl through C9, as in any normal calibration. At the start of the experiment 
itself, a point at El would appear on the display, when the test subject felt he was fixating on the 
point, he would press a key. The test subject was asked to maintain his fixation on the point until it 
disappeared. For this experiment each of the experimental points stayed on the screen for 10 
seconds after the test subject made the keystroke. The keystroke initiated the data recording 
procedure and stored each lookpoint for the 10 second window. Data were recorded only after 
each keystroke to eliminate data points relating to acquisition of the experimental point. This 
allows an estimate of the error based only on the data relating to the static designation of the 
point. 


This procedure was repeated for each experimental point in sequence from El to E5. 
The test subject was recalibrated after the last data point and asked to repeat the procedure using 
the new calibration data. Each of the test subjects performed ten trials of being calibrated and 
then fixating on points El through E5. The test subjects did not receive any feedback on their 
performance and all calibrations were used. Three test subjects were used in the experiment 
representing qualitatively small, medium, and large pupil diameters. All of the other experimental 
conditions were kept constant to determine if pupil diameter had any noticeable effect on the 
data. 

Data Reduction 

Data were recorded as a pair of screen coordinates for each POG. As discussed 
previously, it is important to examine the size of a cluster and the offset of the cluster from the 
desired display point. The size of the cluster was evaluated with three different types of 
measurement. The average value for all of the data points was used as the center of the cluster, 
with the size of the cluster defined as the distance from the center to the individual points. This 
distance was calculated as. (a) the average distance from the center to the individual data points; 
(b) the root mean square (RMS) of the distance from the center to all of the data points; and (c) as 
the maximum distance from the center to any one data point. From observing the data, it was 


42 





noted that in some runs there was a small fraction of the data points that did not seem to be an 
integral part of the cluster. The cause of these outlying points seemed to be related to 
discrimination problems in the oculometer, so when this occurred the data were reevaluated 
based on the 540 out of 600 (90%) closest data points. As will be discussed below, this produced 
a significant reduction in the calculated cluster diameters. The displacement from the cluster 
center to the designated display point was converted to a distance and orientation from the 
designated display point. The magnitude was calculated as the RMS distance of each point from 
the designated display point. 

Because the data recorded were in screen coordinates, it was necessary to transform 
these data into a format that could be related to a physical (cockpit) measure. Three types of units 
were considered: inches, degrees, and miliiradians. Inches were not used because the data 
representing a line of sight value would change based on the distance from the observer from the 
display. Of the two angular measurements, it was felt that the degrees unit could be more easily 
understood. Therefore, following standard visual science practices, all of the data presented in 
this report are in degrees. 

System Performance 

The data presented in this section were collected during the experiment along with 
observations made by the staff at Analytics based on their experience with the oculometer 
system. All of the data collected during the experiment can be found in Appendix B; selected 
pieces of the data are reported in this section. Overall, these data represent the best that can be 
realistically expected from the CORS prototype. 

Best and Worst Case Performance 


Table 4-1 contains the best (smallest) and worst (largest) cluster sizes from the trials of 
Subject 3. The performance of Subject 3 was the best of the three test subjects in the sense that 
the results were the most consistent. The best cluster radii are approximately from a quarter of a 
degree to a third of a degree while the worst does not exceed 0.564 degrees RMS. The radius of 
the extreme values are two to three times the size of the average or RMS radius and this 
relationship holds true to a lesser extent for the 90% data. Because this relationship occurs while 
the average and RMS radii do not significantly decrease when using the 90% data, this leads to 
the conclusion that the data points are highly clustered within their average and RMS radii. If this 
conclusion be true, it is reasonable to expect to be able to develop a filtering algorithm which 
could calculate cluster sizes from a quarter to a half of a degree. This value represents the human 
limitation inherent in an eye-based system (Hallett, 1986). 
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Table 4-1 . Eixcerpt of cluster size data for Subject 3 (in degrees). 
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Figure 4-2 is a graphical representation for the best trial of Subject 3. The plus signs 
represent the five experimental points. The center of the circle is the calculated cluster center. 
The radius of each circle is based on the point furthermost from the cluster center, the .nner rad.us 
being the 540th furthermost point and the outer radius being the 600th point. The center of the 
cluster is very close to the experimental point. Notice, however, that each cluster is located to the 
left of the experimental point, which indicates a small error induced during the calibration 

procedure. 

Average System Performance 

The tabular data in the next two tables represent a composite of all three test subjects. 
These data represent what would be average system performance without recalibrating the test 
subject after a poor calibration. Graphical data are illustrative of runs collected during the 
experiment and were selected on this basis. 

Table 4-2 contains RMS cluster radii for 90% and 100% of the data points for each 
experimental point, for each test subject, and a composite of all test subjects. Table 4-3 lists the 
cluster offset from the experiment point for each point, for each test subject, and a composite of 
all of the test subjects. As a composite the RMS cluster radius was approximately 0.75° with an 
offset of approximately 1.2°. The results for Subject 3 are significantly better than those of 
Subjects 1 or 2. Furthermore, the difference between the 90% and 100% RMS cluster radius 
distance is significantly greater for Subjects 1 and 2 than for Subject 3. Obviously the system was 
having trouble with Subjects 1 and 2, producing data points that were significantly displaced from 
the overall cluster. Our informal observations of the difference between the test subjects noted 
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Figure 4-2. Best result. (Subject 3, 






that the pupil ot Subject 3 is larger than the other too test subjects. Whether this is thei primary 
cause ot test ditterenoes or there is some other reason is unknown. However, this variable should 
be explored lurther to characterize system performance more completely. 


Table 4-2. RMS cluster size data for all test subjects (in degrees). 
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Table 4-3. RMS offset data for all test subjects (in degrees). 
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One ol our experimental observations was that it is much more difficult to discriminate all 
test subject's pupil and cornea in the lower pad ol the FOV than elsewhere and the system 
pedormed much worse in this region. Experimental points E4 and E5 are 
the other three points which are ih the upper FOV. II these values were excluded from the 
composite calculation there would be a significant reduction in the total RMS cluster radius a 
RMS cluster offset. This points to a system pedormance deficiency that should be correct 

future work. 
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Figures 4-3 and 4-4 show two of the trials from the experiment; Figure 4-3 shows an 
average tnal and Figure 4-4 shows one of the worst trials. In Figure 4-3 all of the clusters overlap 
the experiment point with four of the five points falling in the inner ring, showing that 90% of the 
values were in this area. Note that the cluster size is largerthan was shown in Figure 4-2 and that 
the worst points are in the bottom of the FOV. The trend observed in Figure 4-3 continues in 
Figure 4-4 with a much more pronounced effect. The clusters for experimental points E4 and E5 
bear no real relation to those points and the data are spread throughout the display. The other 
three clusters are larger, but the inner 90% of the points are within an acceptable range. The 
offset is larger and the clusters for experimental points Et and E2 do not overlap their respective 
points. Fortunately, results like those shown in Figure 4-4 did not occur often and should be 
correctable by improvements in the hardware and software algorithms used to process the data. 

Correlations between System Deficiencies and Components 

There are three components of the oculometer that seem to be the most likely sources of 
error illustrated in Figures 4-3 and 4-4. Those components are: discrimination, calibration and the 
associated interpolation routines. The variabilities of cluster sizes and extraneous data points are 
most likely related to the discrimination hardware and software. Errors in the calibration process 
are carried through into the interpolation routines which produce the output from the oculometer. 
These errors cause an offset between the cluster center and the desired display point. 

Errors induced by poor discrimination or the complete inability to discriminate either the 
pupil or cornea seem to be the likely reasons for the wide fluctuations in cluster size and the 
random data points that the system produces. The inability of the system to discriminate the pupil 
has been observed regularly in the lower half of the FOV. Informal observations confirm the 
above suggestions. The cross hairs displayed on the video output of the pupil camera designate 
the estimated pupil center. When the fluctuations occur and the test subject is sitting quietly the 
cross hairs jitter between two widely separate locations without any noticeable accompanying eye 
movement, indeed, the jitter is too fast for eye movements to occur. This is most likely caused by 
the EVM system optics or the hardware processor used to calculate the pupil centroid. 

The calibration procedure requires the test subject to fixate on the calibration points in 
sequence. The values recorded at this time are used as data points to bound curves used in the 
interpolation routines. If the values recorded during the calibration procedure are inaccurate an 
offset is induced into the calculated eye point. Figure 4-5 shows the angular offset for the 
Subject 3 from the experiment. In runs 1 , 5, 6, and 1 0, the angular offset is constant and could be 
corrected during the data run by recalibrating one point. When angular offset is spaced over a 
wide arc a specialized calibration routine is required. 
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Figure 4-5. Angular orientation of cluster center from desired display point (Subject 3, Runs 
1 through 1 0). (Where there are less than five arrows, dark arrows are used to indicate 

multiple lines.) 


Figures 4-6 through 4-9 display the coordinates of the calculated eye points for Subject 
3, run 7 and experimental point 1 . The data fluctuate around the center of the cluster. Because 
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Figure 4-6. Eye lookpoint X coordinate (Subject 3, Run 7, Point El). 
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Elapsed Time in Sixtieths of a Second 

Figure 4-7. Eye lookpoint Y coordinate (Subject 3, Run 7, Point El). 

the interpolation routine utilizes the relationship between the pupil and cornea at each calibration 
point any deviation leads to an induced offset which is propagated through a sizeable portion of 
the FOV. The averaged data show how the eye settles on a single point and then slowly drifts to a 
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Elapsed Time In Sixtieths of a Second 

Figure 4-8. Sixty frame average of eye lookpoint X coordinate (Subject 3, Run 7, 

Point El). (The plot is a running average of the raw data in Figure 4-6, using 30 
frames either side of each plotted point.) 

new fixation point. All of this occurred while the test subject focused on the desired display point. 
Also, there is an initial time at the beginning and end of the data where the test subject has not yet 
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settled on a specific point. The calibration routine must recognize this and extract its data from the 
point where the eye has settled to extract a good value for that point. Again the data support the 
conclusion that a more sophisticated calibration routine is required to improve system 
performance. 



Elapsed Time In Sixtieths of a Second 


Figure 4-9. Sixty frame average of eye lookpoint Y coordinate (Subject 3, Run 7, 
Point El). (Averaging is done in the same way as in Figure 4-8.) 


Further Experimentation on System Accuracy 

A second experiment was carried out to examine CORS performance in greater detail. 
The principle behind the experiment was to present stimuli which were sufficiently small to insure 
that the test subject looked directly at them in order to identify the test items. These acuity targets 
require foveal vision. By using acuity targets we insured behaviorally where the test subject was 
looking before responding. By examining those cases in which the test subject responded 
correctly, we could then examine the performance of CORS. Because the test subject had to 
look at the stimulus to identify the item correctly, CORS should report a fixation on the target at 
some time during (or ending slightly after) the stimulus presentation. 

The primary goal of the experiment was to isolate system performance from test subject 
performance. This is possible by means of examining the relations of the possible outcomes for a 
given experimental trial. Figure 4-10 represents a 2 x 2 contingency table and shows the possible 
outcomes for the experiment. When CORS 'missed' the fixation and the test subject correctly 
identified the stimulus (upper right cell of Figure 4-10), represent cases in which the filter settings 
may be too stringent. That is, if CORS were functioning perfectly, there would be no entries in 
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this cell. A second goal is to examine the positioning of the fixation in the area around the 
stimulus in the context of human performance. That is, CORS may report fixations but these may 
or may not be close to the stimulus position. 


CORS 



Correct 

Incorrect 

Correct 

Both 

Correct 

Test 

Subject 

Only 

Test 



Subject 

Incorrect 

CORS 

Only 

Null - 
Neither 
Correct 


Figure 4-10. Illustration of the possible outcomes for the acuity experiment. 


Procedure 

On a given trial, two letters were presented. These were selected randomly from the set, 
ACEFHKMNOTVWXYZ. All letters were used equally often and were paired with all other letters 
randomly. The size of the letters was 5x7 pixels [59 pixels/inch] which equates to an overall 
visual angle of approximately 14 x 19.5 minutes of arc. Because discrimination between letters, 
e.g., E vs. F, will be based on a few pixels, the actual effective stimulus is often smaller than these 
overall dimensions. 

The possible spatial positions of the letters is shown in Figure 4-1 1 . The two letters for a 
trial appeared on a given radius, i.e., positions 1 and 2, 13 and 14, etc. The center of the figure 
represents the fixation point for the start of a trial. The separation between the center fixation 
point, the inner circle and the outer circle was approximately 6°. Because the test subject did not 
know where the letters would occur on a given trial, he was encouraged to fixate the center. 
Indeed, the initiation of a trial was started by the test subject when he felt he was looking at the 
center point. To start the trial, he pressed the space bar on a keyboard and the two letters 
appeared shortly thereafter. 
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Figure 4-1 1 . Schematic layout of potential stimulus positions. 



The duration of display of the two letters was varied. The time periods were 300, 400, 
500, or 600 msec. These durations allowed time for the test subject to move his eyes from the 
fixation point to the position of one or both of the letters. At the end of the exposure duration, 
each letter was replaced by a mask which inhibited post-exposure processing. To indicate his 
response for the inner position, the test subject pressed the appropriate letter key on the 
keyboard followed by a press on the space bar. The space bar response indicated the completion 
of the response for that stimulus position. The letter and space bar sequence was then repeated 
for the outer stimulus position. The test subject could change his response simply by pressing an 
alternative key before pressing the space bar. The computer recorded all responses, including 
multiple responses for a trial. The last letter pressed before pressing the space bar was taken as 
the final response for purposes of scoring. 

The combination of conditions were presented randomly within a block with different 
randomizations of conditions for each block. The horizontal and vertical meridians occurred 
equally often and randomly occupied 40 of the trials. The oblique meridians were presented 
equally often on the other 20 trials. The exposure durations were used randomly, each for 15 
trials. Thus, the meridian by exposure combinations were not necessarily tested equally often 
within a block. New calibrations were carried out before each block of 60 trials. This technique 
permitted evaluation of the calibration procedure by block. 

One test subject was run for four blocks. This individual also participated in the previous 
experiment as test subject number 2. The Masscomp system was used together with the helmet 
mounted oculometer. The helmet was restrained to prevent head movement and resulted in a 
viewing distance of 21 inches. This was done to reduce the number of possible factors 
contributing to the data. 

Results 

The analysis of eye position was accomplished using a special purpose program. This 
program uses the successive lookpoint data obtained and written to file by CORS. The scoring 
procedure incorporates the temporal portion of the filter (10/12) and the spatial setting at a radius 
of 0.316 inches (52 minutes of arc) at the calibration plane. The scoring for the region, however, 
was more relaxed. Circles were drawn around the letter positions so as to be as large as possible 
without overlapping. This yielded region fixation circles of slightly less than 6° in diameter. The 
lookpoint data were then examined manually by stepping through the data and examining the 
calculated POGs after each frame. A fixation which fell within the criterion region circle while the 
letter was being displayed was considered to be coincident with the letter for that trial. 

As indicated in Figure 4-10, there are four possible outcomes for a given trial: Both 
correct (test subject correct with CORS identifying the position), test Subject only correct (CORS 
missed the position), CORS only gets position (test subject incorrect), and Null trials (neither 
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performed correctly). The results for all four of these outcomes for each of the exposure 
durations are presented in Table 4-4. As would be expected, performance on the inner letter was 
high. As exposure duration is increased, performance on the outer letter improved dramatically. 
This is a result of the test subject having time to move his eyes to the outer position while the 
letter is still being exposed The two cases of primary interest are those two in which the letter was 
reported correctly. The upper right cells of Table 4-4 provide the data. 

Calibrations were carried out for each of the blocks of trials. Table 4-5 shows the data 
recast in terms of Blocks. The principal point of this table is to show that Block # 1 has a poor 
calibration. This one block accounted for 68 % of the CORS failures. The two rightmost columns 
in the table provide a ratio of the number of CORS 'misses' (test subject correct) to the total 
number reported correctly by the test subject. 

The results show that CORS is capable of identifying eye position with the head fixed. 
The experiment also indicates that calibration is a crucial factor. Specifically, in Block #1 , the 
CORS inaccuracy ratio is 0.279 which indicates that the system did not perform nearly in parallel 
with the test subject. For the inner letters, the test subject correctly reported 55 out of 60, 
indicating a high level of performance whereas CORS identified only 44 out of the 55 correct. The 
other blocks, with different calibrations, show apparent acceptable performance by CORS within 
the parameters used. 

One purpose of the experiment was to examine the spatiotemporal filter. The extent to 
which fixations are identified regardless of location provides evidence for the viability of the 
spatiotemporal filter settings. The level of performance in most of the runs indicate that the 
spatiotemporal filter performed well, with a high percent of fixations reported. This is represented 
in the table as the two entries, Both Correct and CORS only. Further, CORS reported fixations 
for the outer letter about 30% of the time when the test subject did not report the letter correctly. 
This result was fairly stable across exposure duration, indicating the eyes moved but that the test 
subject could not identify (he letter. Thus, CORS is identifying fixations. 

The identification of the existence of fixations is necessary, but it is not sufficient. It is of 
utmost importance to have accurate positioning of the fixation within the region. This was not 
particularly good. Letter fixation was defined by a circle of almost 3° radius, 6° diameter around the 
letter. Because CORS identifies POGs and fixations, but only generally in the area of the 
stimulus, this is further evidence for the problems associated with the calibration procedures and 
the application of the calibration data to the lookpoint values. This result is analogous to the 
angular displacement results shown Figure 4-5. Further, the present experiment offers the basis 
of a technique to check on the adequacy of the calibration. 
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Table 4-4. Test subject accuracy and CORS accuracy for exposure duration. (Numbers in () are 
for the outer position.) (Cell entries are number of occurrences out of 60.) 


Exposure Duration = 300 CORS 

Correct Irmrect IM. 
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Table 4-5. Summary of acuity experiment. (Cell entries are number of occurrences out of 60.) 
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Experiment on the Effect of Head Motion on System Performance 

One of the goals of Phase II experimentation was to determine how head motion affects 
system performance. The majority of head motion that occurs in a cockpit is rotational with a small 
amount of translation motion. This experiment considered the initial level of system accuracy, 
conducted an evaluation of the accuracy at different orientations and angles of incidence with the 
panels and determined whether system performance degraded during the experiment. The 
emphasis for this experiment was on spatial point designation where the test subject focused on 
successive points for a few seconds each. This produced definite fixations which could be 
correlated to the regions to determine how accurately CORS could reproduce where the test 
subject was looking. 

The experiment was conducted in the mock cockpit, described in Task 1 . The test 
subject was required to look at the calibration points and then at a sequence of regions 
throughout the cockpit and then at the calibration points again. The regions were placed in 
panels 3, 4, 5, 6, 8, 12, and 15 (in reference to Figure 3-15, p 31.). These panels correspond to 
the full spectrum of angular orientations that would commonly be found in a cockpit. Each region 
is a square with the letter A in the center, the length of the edge of the square being one inch. A 
sequence of regions was defined in the planes program around the letter A to evaluate how 
accurately CORS calculated where the test subject was looking. The largest square was six 
inches on a side, which corresponds to 8° of deviation from the center of the A at 21 inches. 
During the experiment each fixation that CORS calculated was written to a file and then reviewed 
by the play back program. Additionally, the output of the scene camera was recorded on a VCR to 
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be played back to show where the oculometer calculated the test subject to be looking. Ten 
experimental runs were conducted using one test subject. 

The results from the head free runs are inconclusive; the majority of the runs showed 
varying levels of degradation of the calibration at the conclusion of each run. These degradations 
were not consistent across the runs, but the majority of the degradations seem to be a result of 
hardware deficiencies of the helmet mounted oculometer as described in Task 1 . On some 
occasions the visor slid as the test subject tilted his head to look at panels 3, 12, and 15. Helmet 
slippage occurred frequently during the runs as the test subject was released or made rapid head 
motions. Additionally, the scene camera would occasionally move during the run, removing the 
only means of independently verifying the results produced by CORS. These problems preclude 
being able to define the performance of the CORS system numerically with respect to head 
motion or panel fixation, but some qualitative observations can be drawn from the data. 

Generally, the system performed as expected for most of the regions. The best 
performance was in panel 8. This was to be expected since this is the panel that contains the 
calibration points and is perpendicular to the test subject's line of sight, i.e., directly in front. In a 
cockpit, it is also the location of many primary instruments. 

CORS system error is an angular displacement from the test subject's line of sight. As a 
result, when the test subject looks at a panel at an oblique angle to the line of sight the linear error 
is magnified as a function of the angle between the line of sight and the panel. This was most 
dramatically shown in panel 15 where the angle of incidence was on the order of 20°. CORS 
usually calculated the test subject to be looking at the background of the panel instead of at a six 
inch region that included the letter A. CORS usually calculated to within two inches of the correct 
position in panels 3, 4, and 12 which had angles of incidence of approximately 70 to 80°. 

The worst results were obtained for panels 5 and 6, which correspond to the left and right 
sides of the cockpit. In both of these panels CORS had difficultly calculating the correct location 
as to where the test subject was looking. Review of the videotape usually showed that the 
oculometer was providing what should be the correct coordinates for A, and the angle of 
incidence was close to perpendicular. The only difference between these two panels and better 
panels, beside the obvious rotation, was that panels 5 and 6 were much closer to the test subject. 
At this point it is unknown whether the problems associated with these two panels are a system 
deficiency in the algorithms or a hardware related problem in either the helmet or the head tracking 
system. Unfortunately, the constraints imposed by the hardware and the conclusion of the 
contract precluded a more thorough examination of the effect of head motion on overall system 
performance. 


60 



Summary of Experimental Issues 


A number of issues were to be studied within the experiments. In retrospect, several of 
these issues were based on questions that were not necessary to study. The most important 
reason for the change in relative importance of the research issues was the availability of 
technologically improved computer hardware in which processing could be performed at a much 
higher rate than anticipated. Other issues included the study of calibration. However, it was not 
possible to modify the calibration routines without having the oculometer manufacturer perform 
this function. 

Some of the proposed issues were studied in the experiments described above. For 
completeness and convenience, we address all the issues identified in the original proposal here 
in summary form. This is done independently of whether the experiments investigated them or 

not. 

A Characterization of System Head Motion Tolerance 

As indicated in the Task 1 equipment discussion, the helmet presents considerable 
problems as currently configured. Whenever there is head movement, there is often helmet 
movement and movement of the associated devices mounted on the helmet. Specifically, the 
helmet may move in relation to the head, the visor may move in relation to the helmet, and the 
scene camera may move in relation to the helmet. The combinations and effects of these variable 
movements defy any system of compensation within the constraints of the contract. 


System Accuracy Relative to Head Motion/Location 

The two Masscomp experiments indicate that tolerable accuracy can be achieved without 
head movement. However, when the helmet is released, CORS performance degrades. The 
combination of the helmet problems and the calibration problems made it difficult to characterize 
the errors in any formal manner. These random variables have also made it difficult to test the 
algorithms involving the integration of head and eye position data. Nevertheless, several 
observations point to potential problems in the algorithms. One of these involves the angle of 
incidence which may cause disruptions in the calculations and relates to accommodating the 
depth of the panel. 

While these results are not as good as one would like, they do not negate the CORS 
concept. They do, however, emphasize the extreme importance of the hardware used and 
indicate that further work is needed to test and develop the system, including the algorithms. 
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Clearly, improving the helmet system would be of considerable benefit as a first step in any such 
future research. 


Optimization of Calibration Procedures 


The EVM provides modified output based on fitting the individual x,y observations to the 
calibration points using the calibration plane. That is, the lookpoints are modified as a result of 
having been passed through the calibration routines. 

Experiments were carried out to characterize the accuracy of the oculometer and the 
CORS system. The findings of these experiments suggest the CORS oculometer accuracy is 
variable and is dependent on, at least, the following variables: 

• Time since calibration, 

• Location in the visual field, 

• Test subject, including the size of pupil and other physiological variables 

It should be noted that the calibration routines are proprietary, and therefore it was not possible to 
modify these routines under this effort. Thus, the accuracy and errors noted are characterized in 
relation to the calibration, but were not corrected. 

Optimization of Eye Position Sample Rate (samples per second) 

The consideration of optimization of the eye position sample rate is based on two 
separate issues. One of these relates to the real time processing capabilities of the CORS 
computer and the other has to do with accuracy or fidelity in capturing the movements of the eye. 

One of the concerns prior to developing CORS was whether a PC could keep up with the 
necessary calculations required for CORS if the data rate occurred at 60Hz. For each new frame of 
data, the computer has to do calculations on up to 12 time frames of data. These calculations 
include integrating the eye and head position data for each lookpoint, averaging a number of 
lookpoints, and possibly looking up a dwell region and recording data. 

Within reason, it is easy to argue that the greater the sample rate, the better the accuracy. 
The duration of a saccadic movement has a range between 20 and 100 msec (Haliett, 1986), 
however, the standard deviation was not reported, and consequently one can only guess about 
the shape of the temporal distribution. (Simply using the middle of the range would put the typical 
duration at 60 msec.) The purpose, however, is not to capture the eye while it is motion but to 
know where it is between movements. Thus, the intent is to use the eye movement as a ’trigger 1 
to determine the end of a fixation and the beginning of the next. Accordingly, the objective is to 
filter this movement out. 
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Different sampling rates have been used in oculometer studies. At 60Hz sampling rate 
the data are sampled about every 17 msec; at 32Hz the data are sampled about every 31 msec. 
These sample rates are approximately two to four times faster than the midrange of the saccadic 
movement times (20-100 msec). The primary effect of a sampling rate change is on selection of 
the appropriate parameter s for the spatiotemporal filter. Stated differently, halving the sample rate 
requires changing the spatiotemporal filter from 10 out of 12 lookpoint frames to 5 out of 6 to 
maintain the similar accuracy relations. 

Neither the oculometer nor the head tracker provides a provision to alter the data sample 
rate. To change the sample rate, we are left with the sole alternative of ignoring some of the data. 
However, such steps were not required, since the preset data rate of 60Hz was processed by the 
CORS computer. 

Trade-off between Data Reduction and System Accuracy 

There are several points in the CORS system where data reduction and system accuracy 
trade-offs could occur. One such point resides in the spatiotemporal filter. Changing the size of 
the spatial area leads to a more rigorous criterion (smaller radius - fewer fixations) or a more lax 
criterion (larger radius - more fixations). Similarly, changing the temporal parameter also relaxes or 
tightens the accuracy, i e , the number of frames (x) required to be within the range relative to the 
total number of frames (n) contained in the POG window. 

A second data reduction routine is based in the recording option , i.e., on and off fixation 
records vs. dwell records (Figure 3-12). This point clearly influences the accuracy of the data 
recorded. Because dwells will often contain fixations with null regions interspersed, the fine detail 
of oculometer data will be lost when only dwells are recorded. 

Both the spatiotemporal filter and the mapping can be changed within CORS, depending 
upon the specific situation. Accordingly, the user has control over the data reduction/ system 
accuracy factor. Further experiments are required, however, to provide appropriate guidelines 
and recommendations sc the end user can make these decisions. 

System Response as a Function of Scan Rate 

This, too, was an issue that was not important after the final selection of the CORS 
computer. Basically, CC RS can handle any eye scan rate. The limitations within the prototype 
were defined by the spatiotemporal filter along with a desire not to drive the recording system too 
hard. Because of the number of words being written to the DFDR and the speed of the CORS 
computer, we intentions ly eliminated dwells shorter than 200 msec. This is not a permanent 
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limitation, however, and can be changed in an operational system, provided suitable substitute 
hardware is available. 


CORS Prototype Demonstration 


A final goal of the CORS effort was to demonstrate the integration of the prototype 
equipment as a functioning system. This demonstration utilized the cockpit mockup to exercise 
the system in a representative environment that provided dynamic eye movements throughout 
defined regions. The primary factor influencing the demonstration protocol was that of verifying 
the proof-of-concept by showing the ability to reconstruct pilot eye movement history from the 
DFDR recorded data to an acceptable degree of fidelity. 

As implemented, the CORS computer performed all data analysis prior to recording the 
output on the DFDR. Accordingly, data actually recorded consisted of eye dwells coded by 
region and time of initiation (delayed slightly to account for the decision criteria of 10 of 12 
acceptable measurements. Thus, the system demonstration involved collecting, reducing, 
coding, formatting, and writing data to the recorder and then retrieving the data from the DFDR. 

Being a research project, the CORS software is implemented to provide flexibility and 
insight to its operation and performance. While the data collection and processing steps are, in 
general, basic and common throughout, the writing step has several alternatives. Processed data 
may be written to disk, to the computer screen, or to a recording device, in this case the DFDR. 
During the early steps of development and testing the data were written to the screen and the 
disk so that they could be analyzed quickly, modified if necessary, and then reanalyzed prior to 
formatting in the FDAU-M and recording on the DFDR. In the final stages of the effort data were 
formatted in accordance with the ARINC standard and written to the DFDR. The CORS results 
discussed previously attest to the capability to collect, process, and write eye history data. 

The ability to write data to and retrieve data from the DFDR was demonstrated in 
conjunction with the CORS computer. A test routine was developed which wrote a series of 
records to the DFDR from the CORS computer through the FDAU-M onto the DFDR. The FDAU- 
M, in addition to formatting and automatically transmitting the record series to the DFDR, also 
generates its own validation records which are written to the DFDR. These parallel data are 
retrieved by reversing the the process, i.e., the CORS computer activates the playback mode to 
the DFDR through the FDAU-M. Comparison of the dual records retrieved via the software 
demonstrates the playback capability of CORS. The software used was tested repeatedly in this 
fashion with the FDAU-M and the DFDR. This effort constituted the prototype demonstration. 
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future directions and potential improvements 


A variety of improvements can be made to the prototype system. Many of these are 
straightforward and. in some cases, the equipment already exists. Generally, to implement the 
improvements would mean replacing the oculometer and improving the helmet mounted sensor 

design. 


The Helmet 

The difficulties with the helmet were documented in the other sections. Clearly, a better 
fitting helmet would be of considerable value. One way to accomplish this would be to use pads 
attached with velcro to adjust the helmet size. Another possibility is to use several sizes with the 
means to transfer the associated sensors from one to the other. Further the attachments 
currently have too much freedom of movement. The visor, the scene camera, and the eye camera 
mounts tend to loosen with no means to tighten the fittings. 

To correct these mechanical aspects is clearly possible and would result in better overall 
performance. 


The Oculometer 

An overall issue central to implementing CORS is obtaining an oculometer that is able to 
track the pupil and corneal reflections reliably. This overall issue can be broken into two separate 
subissues: (a) The technique used in the oculometer and (b) the manner in which the calibrations 
are integrated into the eye position data. Of the two, the calibration issue seems to be the more 
important. In general, a dark pupil tracking scheme, along with a sophisticated algorithm to track 
both the corneal and pupil reflections, appears substantially superior to a bright pupil tracker. 
Unfortunately, our present oculometer uses bright pupil technology with a more rudimentary pupil 
discrimination algorithm There are advantages and disadvantages with either approach, 
however, changes in VLSI and digital processing technology now make the dark pupil approach 
the more viable. In a demonstration provided by a manufacturer, a dark pupil oculometer was 
always successful in tracking the pupil even under conditions in which our bright pupil system 

typically fails. 

General Technology Considerations 

There are two different approaches for identifying the pupil. These approaches to eye 
tracking are known as bright pupil and dark pupil, respectively. The bright pupil approach shines 
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an infrared light into the eye and looks for a "white" reflection from the pupil (as seen by a black 
and white video camera). A "white" reflection will result if the light source and the camera are 
coaxial, a condition generated by the optics. The dark pupil approach utilizes an infrared light off 
axis from the camera and looks for a dark pupil. 

The hardware utilized for the initial pupil and corneal reflection discrimination is largely 
analog in nature. Different approaches involve different timing for conversion of the analog signal 
to digital form. A computer is used to handle some of the more complex calculations associated 
with determining the centroid of the pupil and corneal reflections. 

A bright pupil signal is much greater intensity than a dark pupil signal and it is therefore 
easier for an analog circuit to process it. This approach was developed at a time when analog-to- 
digital conversion was not fast enough to process a video image. Hence, analog circuitry was the 
only viable alternative, and since the bright pupil approach was better suited to analog 
technology, it was the approach used. 

The Bright Pupil Approach 

The present bright pupil oculometer system takes the video input from a head mounted 
camera and outputs eye position data. The eye tracking and calibration portions of the system are 
designed in such a way that one cannot be separated from the other. The present oculometer 
has a complex calibration algorithm that maps eye centroid data to real world coordinates. It is 
difficult, however, to assess the capabilities of these algorithms because problems in 
discrimination are such that is difficult to assess what proportion of the error is attributable to 
discrimination and what proportion of the error is attributable to calibration. In order to get only 
centroid data, we would have to make a modification to the software to permit access to these 
centroid data. This program is proprietary and not available to Analytics. 

The Dark Pupil Approach 

The dark pupil oculometer takes a video image of the eye (either from a remote camera or 
a head mounted system) and produces digital outputs denoting the centroid of the pupil, the 
centroid of the corneal reflection and the diameter of the pupil. These digital outputs can be sent 

to a host computer or they can be sent to another unit which then maps the data onto a video 
image. 

A dark pupil eye tracker is completely different from the present system. Pupil and 
corneal reflection tracking have been reduced to sixteen application specific integrated circuits or 
"custom chip" components. The video signal is digitized to eight bit resolution prior to signal 
processing, thereby reducing signal noise and permitting the use of complex signal processing 
algorithms. The entire system is controlled by software in EPROMs, thereby allowing minor 
systems modifications without substantial changes to the hardware. All of the hardware is 
contained in a single board inside the unit. 
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Dark pupil images involve scanning for objects whose voltages are at or near 0.0 volts 
(i.e., ground or video black level). A purely analog system trying to analyze a dark pupil signal is 
susceptible to signal noise. Analog-to-digital conversion has made significant progress in the past 
few years, and it is now possible to digitize a video signal. When the video image is digitized much 
of the signal noise can be eliminated. Further, digital technology has progressed to the point 
where this digitized information can be processed by elaborate algorithms to eliminate other 
random signals. With these changes in technology it becomes feasible to process a dark pupil 

image more effectively. 

Performance 

In evaluating system performance, special emphasis is placed on the ability to track the 
pupil and corneal reflection under various conditions of eye occlusion and ambient light since 
these two factors have the greatest impact on a CORS system. Neither of these factors can be 
controlled in an operational environment. 

Our current oculometer is particularly sensitive to ambient light. This seems to affect the 
system in two ways. First, bright light causes the pupil to shrink and, hence, reduces the amount 
of light reflected off the retina bounded by the pupil. Second, if the ambient light contains 
infrared light (as is the case with incandescent light) the image of the surrounding skin is 
intensified thereby reducing the contrast between the skin and the pupil reflection. When either 
of these conditions occur, the oculometer invariably has trouble tracking the pupil. Further, the 
oculometer has difficulties with occlusions such as eye lashes and eyelids. These problems are, 
to some extent, inherent to the overall bright pupil approach, but most of it is due to the manner in 
which the oculometer processes the pupil image. The hardware and algorithms, apparently, 
cannot deal with erroneous information adequately. 

A dark pupil oculometer seems to contain solutions to many of the problems associated 
with ambient light and occlusions which would be important in the operational environment. First, 
it can perform in a brightly lit room. Second, it is able to handle eye occlusions better. Such 
results indicate that pupil discrimination algorithms are extremely robust. However, there may be 
problems when shadows are present in the video image. This system can be adapted to track a 
bright pupil if there is such a requirement. 

The Computer 

The CORS computer is capable of performing within the laboratory setting but it has 
marginal capacity to support an operational system. The CORS computer is based on a 32 bit, 20 
MHz CPU. The speed and power of a computer depend not only on the cycle time of the CPU but 
also on a number of other associated characteristics, such as the operating system, the 
input/output facilities, and the instruction set. The operating system, for example, was not 
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designed for on-line data acquisition and processing and does not take full advantage of the 32 
bit architecture. Replacement with an operating system designed for such on-line applications 
would improve overall performance. Similarly, reduced instruction set computers (RISC 
technology) typically show instruction execution two to four times faster than larger instruction set 
machines and would increase the data processing capacity. Finally, the communication between 
the primary CPU on the motherboard and the two other CPUs in the system (on the parallel board 
and the serial board) was not balanced for optimal performance. This imbalance should be 
corrected. 

New Flight Data Recording Technology 

Most of the aircraft parameters currently being recorded on DFDRs are continuous 
temporal data which do not change rapidly. Interpolation between data points is reasonable. Over 
the years, flight data recorder applications have gradually become more sophisticated with more 
variables being recorded. 

Eye movement time histories are dramatically different from the aircraft parameters. The 
eye moves in a discontinuous manner and it moves frequently. Time intervals (dwells) of as little 
as a 100 msec can be important. CORS is capable of capturing these short time frames and thus, 
CORS can generate a large (huge) amount of data relative to other variables being recorded. For 
CORS to be effective in an optimal manner, it will be necessary to incorporate advanced 
technology into new flight data recording systems. 

Write once, read many (WORM) optical disks are one possibility and may well be used in 
future DFDRs. They have the advantage of greatly increased recording speeds, as well as greatly 
enhanced storage capabilities. Use of WORMs would permit expanded data recording in modern 
airliners, including that of pilot eye tracking. 

The Ideal System 


Having discussed some of the possible improvements to the prototype, it is now 
appropriate to consider what an ideal operational system might look like sometime in the future. It 
is entirely within the practical realm to envisage the day when CORS will be an operational device 
used every day in aviation. The components of CORS would look somewhat different from the 
prototype discussed in this report; however, most of the components of this ideal system utilize 
realizable technology. 

The weight of the components would be a consideration. The flight data recorder is 
already on board so that item is not a consideration. However, the CORS computer, the 
oculometer, and the head tracker would need to be added. These items have some considerable 
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weight in the prototype. However, the use of very large scale integration (VLSI) digital circuitry 
along with miniaturization of the other components would permit future development of a light 
weight system. The software would be put in read only memory (ROM) eliminating the need for an 
onboard disk drive. For maintenance purposes, the keyboard, the monitor, and a disk drive could 
be plugged into the system but would not be needed in operation. 

The biggest obstacle is the engineering of the sensor devices to meet with acceptable 
human engineering practices for safety, convenience, and comfort. For both laboratory and 
prototype purposes, a helmet mounted oculometer is acceptable. However, the CORS sensors 
need to be integrated with other devices. For example, commercial pilots currently need and use 
a microphone and an ear phone for communications. Many pilots also wear glasses. For this 
reason alone, it is impractical to wear a helmet. Combined with other considerations such as 
safety, helmet weight, and a probable lack of pilot acceptance of a helmet, it is clear that the future 
operational system would not include a helmet. 

Because many pitots already wear spectacles, it would seem that an approach utilizing eye 
glass frames may be a viable one. A number of items could be integrated with the eye glass frame: 

• The boom microphone could be conveniently attached to the eye glass 
frame. 

• Prescription lenses would individualize the eye glass frames for each pilot. 

• The eye monitor camera which would be light weight and mounted on the 
frame. It would measure light reflected off the eye glass lens just as a half 
silvered mirror would. 

• The head tracker transmitter could also be attached to the glass frame. Use of 
an infrared head tracking system could provide separation of signals for the 
head movements of the pilot and the co-pilot. 

Another advance would include simplified and automated calibration procedures. These 
automated routines could be integrated into the checklist and preflight checkout procedures. 
While this approach would not totally eliminate the need for calibration, it would reduce the steps 
currently required in the prototype. 

Finally, the future operational CORS system might also integrate the OASIS concept. 
OASIS provides eye-voice integration to provide hands free command and control of many or all 
aircraft system functions. 
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CONCLUSIONS 


The overall goal of developing a prototype system to record eye scan data on a flight data 
recorder was accomplished. The present CORS system was built as a laboratory prototype and 
therefore does not meet the performance requirements of an operational system. The 
transformation of CORS from a laboratory prototype to an operational system was beyond the 
scope of this effort; however, the potential capability of CORS was clearly demonstrated during 
the current research. 

The primary function of CORS is the collection and recording of eye scan data, stated in 
terms of positions and instruments scanned in the cockpit. To this end, a number of detailed 
aspects of the prototype system were demonstrated: 

• The integration of the oculometer and head tracker signals so as to be able to 
obtain observations throughout the entire cockpit, 

• The development of data processing routines on a personal computer, 

• The demonstration that a moderately priced personal computer is capable 
(marginally) of performing the necessary processing, 

• The writing to and recovery of data from a commercial digital flight data 
recorder, and that 

• The entire system can be assembled from off-the-shelf components. 

The equipment components were assembled into a functional prototype system with data 
processing algorithms providing the integration. One purpose of a prototype is to demonstrate 
the concept and to identify areas for improvement. Accordingly, there are several suggestions 
discussed here. First we provide a brief summary of the capabilities of CORS which provides the 
background for the improvements and then a discussion of possible future improvements. 

Although CORS does a reasonable job in identifying each dwell region, CORS is currently 
less accurate in areas off the test subject's midline (the longitudinal axis of the mockup cockpit) 
than on the test subject's midline. Performance of CORS is an interaction of concatenating 
individual variables and although we have identified the important variables, we cannot isolate and 
improve the first link in the chain, calibration. Several problems in the components were 
identified: 

• The calibration routines within the oculometer, 

• The reduced capabilities of the oculometer in the lower portion of the field-of- 
view, and 

• The helmet which contributes to sensor instabilities. 
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Several other accuracy issues in the prototype were also identified. For the first, this is simply a 
matter of geometry. In the second case, it is not presently known where the basis of the accuracy 
deficiency exists. These eire: 

• For processing a POG within a plane in which the line of sight is not 
perpendicular to the plane and 

• For processing a POG within planes to the left or right of the test subject. 

The combination and chaining of problems as well as completion of the period of performance has 
precluded testing the head movement/eye position integration more fully. This is certainly an area 
where additional work should be done. However, it is both appropriate and necessary to correct 
the hardware problems first. 
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Glossary 


Harris, Glover, and Spady (1986) developed a number of definitions. The use of terms 

report is intended to be consistent with and parallel to the use by those authors. Additional 
are also defined. 


in this 
terms 


bps. Bits per second, an electronic data transmission rate. 

Dwell time. The time spent looking within the boundaries of a region. * 

Fixation. A series of continuous lookpoints which stay within a circle [of 52 minutes radius at the 
calibration plane]. * 

Lookpolnt. The current coordinates of where the test subject is looking within a single time 
frame. * 

Background region. A region which does not contain an instrument. 

•culometer. A device which measures the lookpoint of a test subject. * 

Out of track. A state in which the oculometer cannot determine where the pilot is looking such 
as during a blink or when the subject’s head movement has exceeded the tracking 

capabilities of the oculometer. [Not applicable in CORS due to head movment tracking, 
but see null region ] * 

Panel. A surface area in the cockpit. A panel may contain instruments or could be, for example 
a window. 

Plane. A three dimensional representation of a panel in CORS. 

Point of gaze. A term parallel to fixation. The point of gaze is determined with reference to the 
EVM calibration plane and has to be converted to a region. It is a moving window 
containing 12 lookpoint frames and spatially represented by a circle of 52 minutes radius 

Point of regard. The lookpoint translated into regions on a plane, e.g., an instrument. 

Region. An area on a panel, the perimeter of which corresponds to instrument boundaries. 

Saccade. The spatial change in fixations. The duration of a saccade is within the range of 20 to 
1 00 msec.] * 

* Indicates definitions found in Harris, et al. (1986). Material in [ ] has been added to the Harris et 

al. definition. 
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APPENDIX B 


Data for individual Subjects, Accuracy Experiment 
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Table B-1. Cluster size data, Subject 1. 
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POINT 


1 00% 


T 90% 1 
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^ 

RMS 
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deg 
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1 

1 

0.8438 

0.9660 

2.6253 

0.7518 

0.8480 

1.4157 


2 

0.4194 

0.5840 

2.5211 

IHSWcTiEFI 

0.3550 

\mvrm 


3 

0.3649 

0.4550 

1.6200 

0.3049 

0.3390 

0.6237 


4 

0.5647 

1.0630 

7.6846 

0.4045 

0.4460 

0.8384 


5 

0.4528 

0.5280 

1.4301 

0.3942 

0.4480 

0.8510 

2 

1 

0.5665 

0.6480 

1.4694 

0.4975 

0.5550 

1.0269 


2 

0.3513 

0.4140 

I 1.1464 

0.3004 

0.3420 

0.6864 


3 

0.6431 

1 .1960 

5.0805 

0.3847 

0.4150 

0.7437 


4 

0.4348 

0.4880 

1.3011 

0.3824 

0.4150 

0.7198 


5 

0.2345 

0.2620 

0.7365 

1 0.2088 

0.2270 

0.3852 

3 

1 

0.8037 

0.8630 

1.8144 

0.7374 

17.3200 

1 .2700 


2 

0.4257 

0.5930 

2.5693 

0.3125 

7.8300 

0.8664 


3 

0.6233 

0.9660 

4.9258 

0.4636 

1 1.3100 

0.9173 


4 

1.0292 

1 .4040 

8.6984 


20.3220 

2.1395 


5 

0.4357 

0.8260 

16.7731 

0.3608 

5.8400 

0.7230 

4 

1 

0.4911 

0.5650 

1.6452 

0.4275 

10.6000 

0.8979 


2 

0.5606 

1.0990 

7.2548 

0.3806 

9.7100 

0.8447 


3 

0.7581 

1 .1560 

4.9673 

0.5177 

13.6800 

1.3580 


4 

0.8948 

2.1680 

13.9147 

0.5331 

12.9200 

1.0206 


5 

0.9291 

1.4000 

3.6071 

0.6458 

20.2300 

3.0370 

5 

1 

0.7775 

0.8110 

1.6628 

0.7302 

0.7530 

1.0021 


2 

0.4122 

0.4700 

1.2651 

0.3653 

0.4080 

0.7166 


3 

0.4253 

0.6120 

3.0650 

0.3297 

0.3640 

0.6319 


4 

0.9309 

1.2540 

5.3858 

0.7000 

0.7950 

1 .7287 


5 

0.4131 

0.4660 

1.0797 

0.3644 

0.3990 
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Table B-2. Cluster size data, Subject 1 (cont.). 


RUN 

POINT 

1 00% 1 

90% 

AMS 
deg . 

RMS 

deg 

MAX 
d^ 

AVG 

deg 

RMS 
deg 

MAX 

deg 

6 

1 

0.5444 

0.6370 

2.8102 

0.4736 

■H 

0.8560 

2 

0.5525 

0.6330 

1.6060 

0.4862 

0.5440 

0.9899 

3 

0.4352 

0.4850 

1 .2655 

0.3883 

0.4230 

0.7130 

4 

1.3097 

1 .7070 

3.6914 

1.0499 

■DFFfZ'J 

3.6107 

5 

0.4884 

0.5950 

2.4800 

0.4131 

0.4570 

0.8082 

7 

1 

0.8217 

0.8650 

1.3882 

0.7816 

0.8210 

1 .1004 

2 

0.2918 

0.3360 

1.0468 

0.2548 

0.2820 

0.4325 

3 

0.3189 

0.3690 

0.9949 

0.2724 

0.2990 

0.6260 

4 

0.5362 

0.6370 

1.9343 

0.4239 

0.4770 

1 .0427 

5 

0.3509 

0.3940 

1 .5609 

0.3094 

0.3350 

0.5858 

8 

1 

0.4285 

0.4860 

1.2763 

0.3770 

0.4160 

0.7541 

2 

0.3351 

0.3860 

0.9187 

0.2963 

0.3370 

0.6102 

3 

| 0.3987 

0.4910 

1.7575 

0.3301 

0.3610 

0.6255 

4 

0.3477 

0.4830 

2.7137 

0.2643 

0.2890 

0.5150 

5 

0.3031 

0.4650 

2.1883 


0.2200 

0.9110 

9 

1 

0.7703 

0.9050 

2.0182 

0.6490 

0.7250" 

1 .6836 

2 

0.5877 

0.8440 

3.0406 

0.4163 

0.4780 

1 .1514 

"~3l 

0.4487 

0.6130 

2.5775 

0.3437 

0.3940 

0.7861 

4 

1.2118 

2.1190 

9.4985 

0.7581 

0.8480 

2.2748 

5 

0.3549 

I 0.4280 

2.4489 

0.2981 

0.3340 

0.6251 

10 

1 

0.5953 

0.6770 

1.8798 

i 0.5205 

0.5670 

0.9872 

2 

0.3238 

0.3770 

1.4044 

0.2765 

0.3050 

0.5678 

3 

0.4898 

0.5570 

1.5244 

0.4280 

0.4700 

0.8506 

4 

0.7568 

0.9760 

3.3415 

0.5849 

0.6710 

1 .6182 

5 

0.4587 

0.5070 

ihb n 

0.4095 

0.4410 

0.7906 
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Table B-3. Cluster size data, Subject 2. 


RUN 

POINT 


1 00% 

T 90% 1 
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|| 

inn 
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1 

1 

0.6472 

0.7360 

2.1 1 1 1 

0.5701 

0.6280 

1 .0806 


2 

0.4799 

0.5700 

2.0448 

0.4050 

0.4540 

0.8952 


3 

0.6170 

0.8800 

6.4741 

0.4731 

0.5400 

1 .2790 


4 

0.6350 

0.7210 

1 .7246 

0.5660 

0.6340 

1 .1 158 


5 

1.0287 

1 .1 980 

3.2725 

0.8835 

0.9840 

1 .8198 

2 

1 

0.3843 

0.4330 

1.1334 

0.3396 

0.3720 

0.6526 


2 

0.3563 

0.4050 

1 .4942 

0.3116 

0.3400 

0.6102 


3 

0.3473 

0.4210 

1 .3300 

0.2900 

0.3330 

0.6815 


4 

0.8560 

1 .01 10 

3.3618 

0.7302 

0.8260 

1 .6295 


5 

0.5371 

0.6520 

2.1 147 

0.441 1 

0.4960 

1 .0341 

3 

1 

0.3698 

0.4270 

1 .0747 

0.3252 

0.3680 

0.6544 


2 

0.4460 

0.6060 

4.1975 

0.3608 

0.3920 

0.6688 


3 

0.4086 

0.4810 

1.5339 


0.3860 

6.7324 


4 

0.7306 

0.8830 

3.2873 

0.6079 

0.6900 

1 .3728 


5 

0.5453 

0.6670 

3.1290 

0.4501 

0.4940 

0.9079 

4 

1 

0.3919 

0.4350 

1.0436 

0.3500 

0.3780 

0.6512 


2 

0.3703 

0.4200 

1.2899 

0.3252 

0.3560 

0.6188 


3 

0.4244 

0.5220 

1 .8144 

0.3504 

0.3970 

0.7708 


4 

0.7802 

0.8920 

3.0573 

0.6833 

0.7470 

1 .2786 


5 

0.4077 

0.4890 

2.1982 

0.3450 

0.3860 

0.6792 

5 

1 

0.3464 

0.3830 

1 .1464 

0.3080 

0.3300 

0.5827 


2 

0.7225 

1 .9990 

13.2215 

0.4014 

0.4430 

0.8181 


3 

0.3635 

0.4200 

0.9940 

0.3184 

0.3600 

0.6625 


4 

0.7211 

0.8560 

2.5621 

0.6165 

0.7080 

1 .3995 


5 

0.3193 

0.3580 

0.8876 

0.2805 

0.3030 
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Tatle B-4. Cluster size data. Subject 2 (cont.). 


RUN 


1 00% 

90% 


POINT 

AVG 

deq 

RMS 

deg 
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deg 
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1 

0.4591 

0.5810 

2.3759 

0.3644 

0.4040 

0.8055 


2 

0.5335 

0.6310 

1 .8121 

0.4492 

0.5020 

1 .0346 


3 

1 .2578 

1 .6040 

4.* 

1 .0319 

1 .2660 

2.5337 


4 

0.7306 
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2.1946 
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1 .1613 
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0.2990 
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2 

0.2927 

0.3480 
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0.2494 

0.2850 
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3 
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2.2013 

| 4 

1 .0373 
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3.1841 
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5 
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0.4740 
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0.35131 

0.3920 
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9 

1 

0.4073 

0.4700 

1.3616 

0.3540 

0.3970 
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2 

0.3843 

0.4400 

1 .3142 

0.3346 

0.3670 
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3 

0.3883 

0.4520 

1.3070 

0.3346 

0.3750 
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4 
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0.4820 
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0.3630 

1.1753 
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Table B-5. Cluster size data, Subject 3. 



RUN 

POINT 


100% 


I 90% 1 

B-i 


■1 

AVG 

dea 

RMS 

dea 

MAX 

Hon 

1 

1 

0.3044 



\WGBM 

m 

0.5141 

2 



■arEFZi 

!■» 


0.4492 

3 

0.2810 

f »BckM. 

BESE 

0.2422 

0.2750 

0.51 91 

4 

0.2751 

0.3110 

IK*BZEE3 

0.2431 

0.2670 

0.4776 

5 

0.3301 


IIWMtM 

0.2873 

ism 

0 6079 

2 

i 

0.2733 

HtiSnEE 

IKE^l 

■»UlcL-] 

P" 0.2660 

0.4524 

2 

I'HTI 1 1 

1MM 

imam 


H SEES*] 

0.4645 

3 


BEEBES 

■*TWI 

B‘HIH 

■agCBBI 

0.6521 

4 

0.3243 

■SKHTHil 

lEsmn 

0.2873 

0.3080 

0.5038 

5 

0.2729 

lk!S£B&] 

0.95571 

0.2444 

0.2640 

( 3.4384 

3 

1 

0.3031 

0.3440 

lEEESO 

0.2652 

0.2910 

0.5074 

2 

0.3234 

«EEcrn 

MflfcEPMI 

0.2900 

0.3140 

0.5277 

3 

0.2927 


BKEEHI 

0.2499 

0.2840 

0.5520 


WlrTOI'I 

M»g|»gf»l 



0.3410 

0.6102 

5 

0.2932 

MjEEEHi 

KKFCT3I 

0.2444 

0.2730 

0.5038 


4 

1 

0.2692 

KEIE1 

UUiil:] 

0.2377 

0.2570 

0.4316 


2 

0.2517 

BHEEIH1 


0.2187 

0.2440 

0.4447 

. 

3 

0.3310 

Mum-mil 

KEBI 


0.3220 

0.5471 

. 


0.3834 

KZEH3I 

WHMH 

ERcEMHI 

0.3740 

0.6693 


— 1 

KkmBI 

0 . 3680 T 

KKFm 

0.2774 

0.3020 

0 5660 


5 _ 

1 

0.2909 

MEEm\ 

w&im 

■tjrcrail 

0.2850 

0.5272 

_ 

2 

0.3157 


1.24211 


0.3030 

0.5430 

_ 

3 

0.3125 

BSEESI 

KEOini 

W&FFU 

0.3020 

0.5209 

_ 

4 

0.3040 

mtimi 

0.8704 

0.2688 

0.2950 

0.5254 

L 


5 

0.2647 

BmctiMiii 

0.8587 

0.2278 

0.2520 

0.4681 
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Table B-6. Cluster size data, Subject 3 (cont.). 


RUN 


1 00 % T 

90% 


POINT 

AVG 

deq 

RMS 

deg 

MAX 

deg 

AVG 

deg 

RMS 

deg 

MAX 

deg 

6 

1 

0.3044 

0.3410 

0.8312 

0.2724 

0.3000 

0.51 73 


2 

0.3107 

0.3440 

0.8344 

0.2787 

0.3010 

0.5123 

3 

0.3202 

0.3580 

1 .0215 

0.2828 

0.3060 

0.5358 

4 

0.3049 

0.3590 

1 .1879 

0.2598 

0.2920 

0.5498 

5 

0.5015 

0.5640 

1.2817 

0.4433 

0.4860 

0.8682 

7 

1 

0.3193 

0.3530 

0.8587 

0.2877 

0.3120 

0.5308 

2 

0.3549 

0.3970 

1 .1004 

0.3162 

0.3450 

58.8104 

3 

0.3432 

0.3800 

0.8298 

0.3116 

0.3410 


4 

0.2932 

0.3230 

1 .1970 

0.2647 

0.2840 

0.4623 

5 

0.2823 

0.3190 

0.7090 

0.2503 

0.2780 

0.4920 

8 

1 

0.2656 

0.3040 

0.7284 

0.2327 

0.2600 

0.4785 

2 

0.2810 

0.31 70 

0.7653 

0.2476 

0.2720 

0.4907 

3 

0.2841 

0.3230 

1.2975 

0.2503 

0 . 277 o 1 

0.4799 

4 

0.2968 

0.3440 

0.9827 

0.2557 

0.2860 

0.5525 

5 

0.2963 

0.3280 

0.9566 

0.2647 

0.2840 

0.4772 

9 

1 

0.3035 

0.3370 

0.8456 

0.2679 

0.2880 

0.4993 

2 

0.2692 

0.3230 

1.1487 

0.2273 

0.2550 

0.4812 

3 

0.4032 

0.4350 

0.9710 

0.3671 

0.3870 

0.6192 

4 

0.3148 

0.3580 

0.9236 

0.2774 

0.3060 

0.5606 

5 

0.3067 

0.3390 

0.9701 

0.2760 

0.2970 

0.4835 

1 0 

1 

0.4542 

0.5010 

1.2921 

0.4077 

0.4400 

0.7392 

2 

0.2787 

0.3190 

0.8808 

0.2422 

0.2670 

0.4821 

3 

0.2620 

0.2990 

0.8145 

0.2291 

0 . 2530 ’ 

0.4578 

4 

0.2819 

0.3420 

0.9945 

0.2363 

0.2740 

0.5809 

5 

0.3928 

0.4370 

1.0752 

0.3513 

0.3830 

0.6571 
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Table B-7. Cluster offset data, Subject 1 . 


RUN 

POINT 

RMS 

RUN 

RMS 

1 

1 

0.871 


2 

1 .256 

■ 

3 

0.273 


4 

0.691 

■ 

5 

0.446 


2 

1 

1 .679 

2.662 


2 

2.31 1 



3 

3.297 



4 

3.314 



5 

2.326 


3 

1 

1 .414 

1 .06 


2 

0.889 



f 3 

0.356 



4 

0.243 



5 

1.625 


4 

1 

1.574 

1 .155 


2 

1 .552 



3 

0.29 



4 

0.1 1 2 



5 

0.941 


5 _ 

1 

0.965 

0.824 


2 

1 .1 78 



3 

0.81 



4 

0.338 



5 

0.555 



RUN 

POIN1 

RMS 

RUN 

RMS 

6 

1 

1 .3453 

2.0796 


2 

1.0166 



3 

2.1792 



4 

1 .979 



' 5 

2.2654 


7 

1 

0.4875 

0.91 28 


2 

1.1604 



3 

0.7667 



4 

1 .0675 



5 

0.9255 


8 

1 

0.5484 

0.8217 


2 

0.9426 



3 

ro.6733 



4 

0.8163 



5 

1.0323 


9 

1 

1.022 

1 .2127 


2 

1.5059 



3 

1.2867 



4 

1.8757 



5 

1.422 


10 

1 

1 .2642 

0.7221 


2 

0.7581 



3 

0.1033 



4 

0.5444 



5 

0.3572 



82 























I 


CO 

GO 






CD 










CO 





ro 





i 

RUN 

CD 


CO 

ro 



1 

CO 

ro 




Co 

ro 

1 


l 

CO 

ro 

i 


1 

CO 

ro 

i 

POINT 

o 

H 

o 

o 

B 

H 

CD 


o 

H 

■ 

B 

o 

o 


o 

CO 

o 

o 

o 

4^ 

- 

- 

- 

ro 

33 

00 


CO 

ro 

o 

o 


ro 

CD 

"0 

o 

co 

CD 

CD 

CO 


4* 

CD 

CD 

ro 

O 

CD 

-0 


io 

Sal 

CD 

CD 

CD 

O) 

CO 

o 

CO 

co 
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DO 

(D 


CD 
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co 

CD 

CO 

*^1 

CD 


CD 

CD 
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00 

co 

C/> 

CO 

CO 

CD 

■vi 

CO 


4* 

-vj 

CD 

CD 

— *• 

00 

CD 

IS 

CD 


00 
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CD 
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CO 
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Table B-8. Cluster offset data, Subject 2. 














































































Table B-9. Cluster offset data, Subject 3. 
































































































































































Table B-1 1 . Angular offset data, Subject 2. 


RUN 

POINT 

OFFSET 

ANGLE 

deq 

AVG 
OFFSET 
... deg 

1 

1 

1 9.02° 

1 73.30° 


2 

0.32° 



3 

29.75° 



4 

1 1.24° 



5 

0.18° 


2 

1 

62.27° 

23.52° 


2 

hib 



3 

15.83° 



4 

125.38° 



5 

72.92° 


3 

1 

3.35° 

1 18.54° 


2 

27.73° 



3 

30.99° 



4 

3.77° 



5 

65.87° 


4 

1 

15.71° 

94.83° 


2 

5.01° 



3 

4.23° 



4 

1 2.36° 



5 

28.90° 


5 

1 

103.67° 

51.23° 


2 

58.85° 



3 

4.05° 



4 

160.62° 



5 

4.57° 



RUN 

POINT 

OFFSET 

ANGLE 

deq 

AVG 

OFFSET 

deg 

6 

1 

8.33° 

150.06° 


2 

51.42° 



3 

11.44° 



4 

29.09° 



5 

19.22° 


7 

1 

36.25° 

22.76° 


2 

25.85° 



3 

1 1 .80° 



4 

89.17° 



5 

66.96° 


8 

1 

4.40° 

1 19.72° 


2 

55.88° 



3 

1 1.55° 



4 

107.31° 



5 

35.43° 


9 

1 

15.45° 

133.75° 


2 

49.27° 



3 

31.70° 



4 

22.82° 



5 

1 19.26° 


10 

1 

2.23° 

105.35° 


2 

15.28° 



3 

36.14° 



4 

3.33° 



5 

45.85° 
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Table B-12. Angular offset data, Subject 3. 











































































APPENDIX C 


Oculometer Survey 


88 



Table C-1 . Oculometer Manufactures and models by characteristics of each device. 


OCULCMETER 

MAKE 


COST($) I HEAD 1 ACCURACY I TEMPORAL I 

RESTRAINT RESOLUTION INTRUSIVE 


ASL 1998 
< 1 > 

HONEYWELL 


150.000 
to 

175.000 

400.000 


r W/IN 20* 
OTHERWISE 

2' y 


16.67 ms 


RACKMOUNTED 
CAN IMPLEMENT 
HELMET 
HELMET 
MOUNTED 


NAC Eyemark V 


14,300 I HEADSET 1* 


16.67 ms 


GOGGLE 

MOUNT 


MICROMEASUREMENTS 
SYSTEM 1200 

| 

MICROMEASUREMENTS 

SYSTEM 1200 



DENVER RESEARCH 
INSTITUTE , 

EYEGLASSES 

ASL 1996 <1> 


ISCAN RK426 
<3> 


CHIN 

10,000 RESTRAINED 1* W/IN 20* 


10.000 NONE 3*4* 


16.67 ms TABLE MOUNT 


100.000 

+ 30,000 w/ NONE 
head movement _____ 


16.67 ms 


UNKNOWN 


16 67 ms 


16 67 ms 


HELMET 

MOUNT 

GLASSES 

MOUNTED 

RACK OR " 
HELMET 
MOUNT 
FIXED 
OR 

HELMET 


SST EYETYPER 300 


8.000 I NONE I 2*-3* 


16.67 ms 


UNINTRUSIVE 


UNIVERSAL 
INTRAM CO. 
OFTALMOGRAF 

FORWARD TECHNOLOGY 
<4> 


MOUTH 

13,000 RESTRAINED <5* 

CHIN 

60,000 + RESTRAINED V 


HELMET/ 

HEADSTRAP 

MOUNTED 

CHIN REST 


STOELTING CO. 


10,000 NONE 


CHIN REST 





































































Table C-1 Oculometer Manufactures and models by characteristics of each device. (Cont.) 


OCULOMETER 

MAKE 

EYE MOVEMENT 
W/ RESPECT 
TO HEAD 

HEAD MOVEMENT 
W/ RESPECT 
TO CABIN 

HEAD/BODY 

TRANSLATION 

AMBIENT LIGHT 
CONSIDERATIONS 

VIBRATION 

STABILITY 

CONDmONS 

SIZE/WIEGHT 

CALIBRATION 

TIME/ 

EXPERTISE 

CALIBRATION 

PATTERN 

ASL1998 

<1> 

J— 

20* ANY DIRECTION 

I HACKING MIRROR 
ORPOLHEMUS 

1 CUBIC INCH 

MODERATE 
BRIGHT LIGHT 
PROBLEM 

MAXIMAL CAN BE 
MINIMIZED WITH 
HELMET 

HELMET 10 OZ 
UNIT FAIRLY 
LARGE 

MAXIMAL 
LONGER THAN 

9PTS 

HONEYWELL 

FULL RANGE 

FULL RANGE 

1 CUBIC FOOT 

MODERATE 
BRIGHT LIGHT 
PROBLEM 

MINIMAL 

HELMET 4 LBS 
RACK HUGE 

MINU 1 ES 
MINIMAL 
SEVERAL 

UNKNOWN 

NAC Eyemark V 

MICROMEASUREMENTS 
SYSTEM 1200 
<2> 

MICROMEASUREMENTS 
SYSTEM 1200 

< 2 > 

DENVER RESEARCH 

vv Lcr l -nwH 1 

22* UP-DOWN 

FULL USES 

POLHEMUS MAGNETIC 
SENSOR 

FULL RANGE 

MINIMAL 

MODERATE 

HELMET 17 LBS 
RACK 8 LBS 
MODERATE 

MINU 1 to 

MAXIMAL 15 
MINUTES 

UNKNOWN 

20* ANY DIRECTION 

NONE 

NONE YET 

MINIMAL 

MINIMAL 

SYSTEM 35 LBS 
17*x 16'x 5* 

MINIMAL 

SEVERAL 

uimi rrtc 

9 PTS 

20* ANY DIRECTION 

HULL POLHEMUS 
HEAD TRACKING 
AVAILABLE 

FULL RANGE 

MINIMAL 

MINIMAL 


■ 

9PTS 

INSTITUTE 

EYEGLASSES 

15* ANY DIRECTION 

NONE 

NONE 

MINIMAL 

MINIMAL 

SEVERAL OUNCES 


NONE 


20* LEFT-RIGHT 
25* UP 

5* DOWN i 

TPAH^IKJ^ UIDDAO 





ASL 1996 <1> 

l rlMV/NTw MlnnOH 

ORPOLHEMUS 

2 CUBIC INCHES 

MINIMAL 

[ MAXIMAL 

HUGE 


9 PTS 

ISCAN RK426 
<3> 

15* ANY DIRECTION 

FULL POLHEMUS 
HEAD TRACKING 

FULL BODY 

MINIMAL 

MODERATE 

RACK 12 LBS 
19" x 5.5* x 16* 

1 

5 PTS 

SST EYETYPER 300 
UNIVFRSAI 

45* ANY DIRECTION 

rn* j rrT Dl/^LfT 

NONE 

NONE 

MINIMAL 

MAXIMAL 

20 LBS 

MODERATE 5-10 
MINUTES 

2 PTS 

VI Yl v CnOnU 

INTRAM CO. 
OFTALMOGRAF 

oO LtrT-RfGHT 
40* UP-DOWN 

NONE 

SUGGEST USING 
CAMERA 

NONE YET 

■ 

MINIMAL 

HELMET 3 LBS 
BAR MOUNT 7 OZ 

MINIMAL [ 
SEVERAL 

UIMI ITCC 

UNKNOWN 

FORWARD TECHNOLOGY 

<4> 

25* ANY DIRECTION 

A t* A MV 

NONE 

1 CUBIC CM 


MODERATE 

HUGE 

MINIMAL 

SEVERAL 

uimi rrcc 1 

UNKNOWN 

STOELTING CO. 

4t> ANY 

DIRECTION 

NONE 

NONE 

HSSSISHi 

MAXIMAL 

FAIRLY SMALL 

Hi 

5 PTS 




































Table C-1 . Oculometer Manufactures and models by characteristics of each device. (Cont.) 


OCULOMETER 
MAKE 


SERIAL 

CAPABILITY 


PARALLEL 

CAPABILITY 


ANALOG CAPABILITY 
MAXIMAL 


POWER 
REQUIREMENTS 


# OPERATORS 
REQUIRED 


PRESENTLY 
USING 
SYSTEM 


ANTICIPATED 

UPGRADES 


CO 


ASL 1998 

(BAUD RATE) 
NONE 

20 BITS 
10BITSHORIZ. 
10 BITS VERT 

+/- 5VDC 

120 VAC 
60 CYCLE 

1 

NAVAL TRAINING 
CENTER 
ORLANDO, FL 
INFORMATION 

IMPROVE HELMUT 
SYSTEM DESIGN 
& REDUCE SIZE 
ANY 



HONEYWELL 

CAN EASILY 
BE ADDED AT 

16 BITS 1 

9 WORDS 

EASY TO ACCOMMODA! b i^OVAU 

CAN BE MODULATED 60 CYCLE 

TO 400 HZ ALSO CAN USE 400 CYCLE _ 

AT LEAST 2 

UNAVAILABLE 
c r-rM t cr;F 

CUSTOMIZATION 

NAC Eyemark V 

any baud 

9600 

NONE 

+/-5VDC 

12VDC 

2 FOR CALIB. 

1 TO OPERATE 

OF 

OPHTHALMOLOGY 
UN1V OF 

NONE 

REDUCE SIZE 

MICROMEASUREMENTS 
SYSTEM 1200 

NONE 

9 BfTS HOHIZ. 
9 BITS VERT. 

+/- 5VDC 
TO 

+/- 10VDC 

120 VAC 
60 CYCLE 

1 

ALABAMA & 
OHIO STATE 
i imi\/ r£ 

IMPLEMENT HELMET 

<2> 

MICROMEASUREMENTS 

SYSTEM 1200 

NONE 

9bllo ruriL nncn 

9BrTSHORIZ. 

9 Brrs VERT. 
RAW DATA 

+/-5VDC 

TO 

+/- 10VDC 

120 VAC j 

60 CYCLE 

1 

n rv“\n r*Al ID 

UNIV . Ur 
ALABAMA & 
OHIO STATE 
NATIONAL 

NONE 

INFORMATION 

<2> 

DENVER RESEARCH 
INSTITUTE 

300 

10 bits -1 

5 BITS HORIZ. A VERT 
RAW DATA 

NONE 

.'. l ' S k » -A 

IRIsiSH 

■ 



UNAVAILABLE 
REDUCE SIZE & 

EYEGLASSES 
ASL 1996 <1> 

NONE 

20 BITS 
10 BITS HORIZ. 

+/- 5VDC 

AT LEAST 2 

ANALYTICS. INC. 

IMPROVE SYSTEM 


10 BITS VERT 

6 LINES ANALOG | 

+/- 5VDC 


1 w/ 

VARIOUS MEDICAL 
UNITS 

CUSTOMIZED FOH 

OVERALL 

IMPROVEMENT 

ISCAN RK426 

NONE 

ppoH 

■ 

CONSIDERABLE 

TRAINING 

' 1 \AJ/ 

lIVl r M\-/ V EJY1 ui » I 

POINT OF GAZE 


9600 

pn 

NONE 

■ 

lilluHli I m 

SPECIFIC 
HANDICAPS 
STEVE RUDOLF 

SYSTEM FOR NAVY 
IMPROVE CHIN 

UNIVERSAL 
INTRAM CO 

NONE 

■ 

+/-12VDC 

OR 

24VDC 

■ 

i 

BETH ISREAL 
MED CENTER 
nnnnKS 

RESTRAINT DESIGN 
EXPAND 

OFTALMOGRAF 
FORWARD TECHNOLOGY 

NONE 


‘ ' 4 +/- 5VDC LINES 

H. V, Hvel, Vvel 

■ 

1 

W/ TRAINING 


MARKETABILITY 
IMPLEMENT FlbfcH 

<4> 

STOELTING CO. 

NONE 

mm 

+/- 5VDC 

TO 

+/- 10VDC 

mM 

2 


OPTICS HEAD MOUNT 





















Table C-1 . Oculometer Manufactures and models by characteristics of 


each device. (Cont.) 


OCULOMETER 

MAKE 

“ “point 

OF 

CONTACT 

IB 

■Ball 

IBB 

JOSE VALEZ 
335 BAIR HILL RD 
WALTHAM. MA 02154 

617-690-5100 

9 MONTHS 

HONEYWELL 

JOHN BAHR 
MINNEAPOLIS, MN 

612-542-5837 

1 YEAR 

NAC Eyemark V 

CAREY CLAYTON 
820 S. MARAPOSSA 
BURBANK. CA 91506 

203-668-4803 

30 DAYS 

MICROMEASUREMENIS 
SYSTEM 1200 
<2> 

KEITH SHERMAN 
1921 HOPKINS ST 
BERKELEY, CA 94707 

415-542-0125 

120 DAYS 

MICROMEASUREMENTS 
SYSTEM 1200 
<2> 

SAME AS 
ABOVE 

Same " 

AS 

ABOVE 

120 DAYS 

DENVER RESEARCH 
INSTITUTE 
EYEGLASSES 

GEORGE RINARD 
303-871-4370 

303-871-4370 

3 MONTHS 

ASL 1996 <1> 

JOSE VALEZ 
(SAME AS ABOVE) 

SAME 

AS 

ABOVE 

9 MONTHS 

ISCAN RK426 
<3> 

RIKKI RAZDEN 
755A CONCORD AVE 
CAMBRIDGE. MA 0223a 

617-868-5353 

4-6 WEEKS 

SST EYETYPER 300 

GARYKILANY 
5011 BAUM BLVD 
PITTSBURGH. PA 15123 

412-682-0144 

30 DAYS 

UNIVERSAL 
INTRAM CO. 
OFTALMOGRAF 

HENRY MICHEAL 
P.O. BOX 1915 
3EMMING. NM 88031-1915 

505-546-8205 

6 WEEKS 

FORWARD TECHNOLOGY 
<4> ■ 

WARREN WOOD 
8652 MAGNOLIA SU. 52 
SANTEE. CA 92071 

619-258-8789 

18 MONTHS 

STOELTING CO 

CHARLES SOOlfTEN 
1350 S.KOSTNER AVE 
CHICAGO.IL 60628 

312-860-9700 

4 MONTHS 


FOOTNOTES 

<1> ASL ocubmetere can be adapted for head mounted 

optics witfi helmet. 

<2> Micromeasurements System 1200 yields 
raw eye position data This data can be 
input b Micromeasurements System 7000 
for proprietory processing. System 7000 is an 
IBM AT compatible with propie lory software 
Software abne can be purchased for $4,000. 

<3> ISCAN System RK4 1 6 yields raw eye position data 
This data can be foput b ISCAN System RK520 
for proprietary processing. 

<4> Forward Technologies manufactures and develops 
eyetrackers utilizing and perfecting technology 
initiated at Stanford Research Institute (SRI) 

This work was headed by: 

Dr. Hewitt Crane EK-1 50 
Menb Park, CA 9402S 
415-326-6200 

<S> Raw eye position data as folfows: 


PMpiDiam.- 

9 bits 

Pupi horiz- 

9 bits 

Pupivert- 

8 bits 

Comeal reflec. honz - 

9bits 

Comeal reflec. vert - 

8 bits 

Pupil diam. pos sel - 

1 bit 

Data storage- 

1 bit 







































